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FOREWORD 

This  volume  is  a  contribution  to  psycho-technology.  Mr. 
Moore  has  made,  in  the  following  experimental  and  statistical 
study,  the  first  approach  toward  a  scientific  solution  of  the  problem 
of  placement  of  engineering  graduates  in  a  great  American  indus- 
trial organization. 

It  is  a  bold  and  novel  undertaking,  characteristic  of  a  young 
century,  to  essay  the  application  of  the  principles  and  technique 
of  scientific  method  to  the  solution  of  human  problems  in  industry. 
Yet  it  needs  no  rare  wit  to  see  that  these  very  problems  of  human 
adjustment  must  hold  during  the  twentieth  century  the  focus  of 
study  and  invention  which  during  the  nineteenth  century  were 
chiefly  concentrated  on  the  problem  of  perfecting  the  mechanics  of 
manufacture.  Human  engineering  is  destined  to  a  development 
comparable  to  that  experienced  by  mechanical,  chemical,  and  elec- 
trical engineering,  when  they  first  began  to  draw  largely  on  the 
lustily  growing  sciences  of  chemistry  and  physics.  Thanks  to 
the  foundations  laid  by  these  sciences,  modern  industry  has  a 
highly  developed  technology  of  materials  and  processes.  It  asks 
now  for  a  technology  of  human  nature. 

This  study  in  principles  and  practices  of  personnel  selection  is 
not  a  popular  treatise.  But  its  appeal  will  be  felt  by  at  least  three 
types  of  readers:  the  thoughtful  leader  of  industry  who  ponders 
the  trend  of  scientific  experimentation  on  human  problems ;  the 
young  engineering  graduate  who  is  debating  in  his  own  mind 
whether  to  become  a  designer,  a  manager,  or  a  salesman ;  and  the 
psychologist  who  is  watching,  with  some  apprehension,  the  move- 
ment to  put  into  practical  use  the  tools  he  has  forged. 

W.  V.  Bingham, 

Division  of  Applied  Psychology, 

Carnegie  Institute  of  Technology. 


in 
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PREFACE 

This  report  is  a  description  of  a  specific  study  on  a  practical 
problem.  Certain  principles  and  practices  are  proposed  as  conclu- 
sions from  this  study.  The  practical  problem  was  to  determine 
methods  and  means  for  selecting  young  engineers  just  being 
graduated  from  college  and  placing  them  in  the  type  of  work 
which  they  could  do  best  in  a  large  electric  manufacturing  com- 
pany. The  outstanding  feature  of  the  problem  was  to  select  the 
men  best  qualified  to  develop  into  sales  engineers;  that  is,  en- 
gineers who  meet  the  public  and  sell  electrical  machinery.  The 
study  was  made  primarily  to  solve  the  practical  problem;  and 
then  the  principles  underlying  the  solution  were  formulated  in 
order  that  they  might  be  applied  to  further  problems.  In  other 
words,  the  research  was  made  for  service,  in  the  belief  that  it  was 
possible  to  give  service  and  at  the  same  time  make  a  contribution 
to  science. 

As  a  practical  investigation  the  work  is  already  giving  service. 
The  results  already  obtained  on  one  hundred  seven  engineers 
employed  by  the  firm  during  the  first  year  that  the  new  methods 
recommended  here  have  been  used,  are  very  similar  to  the  results 
obtained  on  the  group  studied  in  this  research.  Assuming  that 
the  executives  classified  the  men  correctly  at  the  end  of  a  year, 
we  can  say  that  the  results  of  the  test  alone  would  have  classified 
seventy  per  cent  of  the  men  correctly  on  the  day  that  they  were 
employed.  Beginning  with  June,  192 1,  the  company  will  use  the 
test  and  other  methods  recommended  here  as  definite  aids  in  clas- 
sifying and  placing  graduate  engineers  whom  they  employ. 

Progress  in  psychology,  as  in  any  other  science,  has  been  made 
by  specialization  and  intensive  study  on  a  specific  problem.  Quite 
often  in  science  both  the  motive  and  the  opportunity  has  been  a 
practical  problem  in  industry.  The  growing  realization  that  the 
human  factor  is  a  most  important  one  in  industry  has  given  both 
motive  and  opportunities  for  research  on  specific  problems  in  psy- 
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chology,  the  science  of  human  behavior.  The  business  world  is  ap- 
pealing to  psychology  as  a  science  to  solve  specific  problems.  Some 
psychologists  have  already  brought  forth  and  exploited  the  facts, 
the  technique,  and  the  point  of  view  of  their  science,  in  the  hope 
that  the  science  might  be  advanced  as  a  science  as  well  as  be  made 
to  serve  man  in  utilitarian  ways.  For  accomplishing  both  pur- 
poses, the  first  and  most  important  principle  is  that  one  definitely 
narrow  and  specific  problem  after  another  must  be  attacked  and 
solved  to  contribute  its  portion  to  the  cumulative  content  of  the 
science. 

This  report  of  a  study  on  such  a  specific  problem  has  been 
written  in  a  comparatively  brief  and  concise  form.  It  is  intended 
for  practical  use ;  but  also  it  is  intended  to  be  read  in  its  entirety 
only  by  those  who  understand  the  language  of  psychology  and 
statistical  methods.  The  original  tables  of  data  are  not  presented ; 
and  much  of  the  explanation  is  omitted  which  might  be  included 
in  an  exhaustive  treatise.  The  tables  and  all  other  details  of  the 
study  are  on  file  with  the  Division  of  Applied  Psychology,  Car- 
negie Institute  of  Technology,  and  are  available  for  all  who 
may  wish  to  examine  them.  However,  it  is  believed  that  the 
report  is  sufficiently  complete  to  enable  those  who  have  an  in- 
telligent interest  in  such  a  problem  to  grasp  what  the  study  has 
to  give. 
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PERSONNEL  SELECTION  OF  GRADUATE 
ENGINEERS 

The  Differentiation  of  Apprentice  Engineers  for 

Training  as  Salesmen,  Designers  and 

Executives  of  Production 

PART  I.     THE  PROBLEM 

I.     The  Practical  Problem 

The  Westinghouse  Electric  and  Manufacturing  Company 
each  year  employs  about  three  hundred  engineers  who  have  just 
been  graduated  from  technical  schools  and  colleges.  All  these 
men  have  had  practically  the  same  training;  but  they  must  be 
differentiated  and  placed  in  different  lines  of  engineering  accord- 
ing to  their  special  interests  and  aptitudes.  The  aim  is  not  to 
discover  which  engineers  are  best  for  any  particular  kind  of 
work ;  for  a  few  very  capable  men  might  be  best  in  all  lines ; 
but  the  aim  is  to  discover  in  what  line  of  work  each  engineer 
will  function  most  efficiently  and  satisfactorily.  That  is,  the 
desired  end  is  to  have  each  member  of  the  organization  so  placed 
that  he  is  able  to  render  his  greatest  service  to  the  Company  by 
having  the  greatest  opportunity  to  develop  his  capacities  and 
promote  himself  in  the  line  of  work  which  he  most  enjoys.  The 
practical  problem  is  to  determine  methods  and  means  for  select- 
ing and  placing  young  engineers  in  the  type  of  work  which  they 
can  do  best. 

For  selecting,  differentiating,  and  properly  assigning  grad- 
uate engineers  to  the  various  phases  of  engineering  and  sales- 
manship, five  methods  or  means  and  combinations  of  them  are 
possible,  namely : 

i.    Grades  made  in  the  technical  schools; 

2.  The  grades,  try-outs,  and  observations  secured  during  the 
year  of  preliminary  training  in  the  firm's  educational  de- 
partment ; 


2  BRUCE  V.  MOORE 

3.  Rating  scales ; 

4.  Interests  and  desires  of  graduate  engineers ; 

5.  Psychological  tests. 

The  second  and  fourth,  interests,  and  observation  during  train- 
ing are  the  ones  emphasized  at  present;  and  in  fact  they  are  the 
best  methods,  at  least  until  new  methods  are  evaluated.  How- 
ever, these  methods  are  not  entirely  satisfactory  nor  ideal;  for 
the  interests  or  choices  of  the  men  are  uncertain  guides  and  are 
not  always  identical  with  the  needs  of  the  company;  and  often  the 
graduate  engineers  cannot  decide  themselves  what  work  they 
would  prefer.  The  process  of  trying  out  under  observation  dur- 
ing training  can  be  done  only  after  a  certain  amount  of  selec- 
tion and  differentiation  has  been  made,  and  the  Company  has 
more  or  less  committed  itself  in  regard  to  its  future  engineers 
and  salesmen.  The  training  could  be  made  more  definite  and 
intensive  if  it  were  known  from  the  beginning,  to  just  what  line 
of  work  the  graduate  engineers  should  be  assigned.  Moreover,  a 
more  or  less  definite  quota  is  needed  for  each  line  of  work,  and 
those  men  should  be  selected  from  the  technical  schools  who 
would  most  exactly  be  the  number  required  for  each  line  of  work. 
Finally,  but  not  least  important,  the  method  of  trying  out  under 
observation  fails  to  reveal  for  what  the  graduate  engineer  is  best 
fitted;  and  at  best,  there  is  often  an  uncertainty  or  question  as 
to  what  work  should  be  assigned.  To  meet  the  conditions  just 
described,  the  first  and  second  methods  are  analyzed,  but  atten- 
tion is  concentrated  upon  the  development  and  application  of  the 
third,  fourth,  and  fifth  methods  named  above,  that  is,  rating 
scales,  survey  of  actual  interests,  and  psychological  tests. 

II.     The  Psychological  Problem  Involved 

Broadly  speaking  the  psychological  problem  is  one  of  indi- 
vidual differences  relative  to  capacity  for  doing  specific  kinds  of 
work.  The  previous  policy  was  to  classify  and  assign  or  place 
the  engineers  according  to  their  interests,  the  practice  being 
limited  somewhat  by  the  openings  or  needs  in  the  Company  and 
by  the  general  impressions  which  the  executives  had  of  the  young 
engineers.     The  big  question  raised  was  whether  the  men  were 
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being  placed  where  they  could  function  most  efficiently.  Were 
their  interests  a  reliable  criterion  of  their  ability?  This  leads  to 
the  fundamental  psychological  questions:  (i)  Is  marked  ability 
in  a  particular  line  of  work  only  general  ability  or  intelligence 
directed  by  interest?  (2)  Are  there  different  kinds  of  intelli- 
gence? (3)  Is  there  one  thing  called  general  intelligence,  and 
additional  abilities  which  are  special  and  which  vary  independ- 
ently? (4)  Is  interest  an  indication  of  a  special  ability  in  a 
particular  line  or  work?  (5)  Are  there  only  special  abilities, 
not  necessarily  correlating  with  a  general  average  of  these, 
which  is  called  general  intelligence?  (6)  Does  success  in  a 
particular  occupation  depend  upon  purely  intellectual  ability,  or 
are  the  more  or  less  vaguely  denned  personal  traits  the  more  im- 
portant for  success?  (7)  When  it  is  learned  which  of  these 
factors  are  important,  how  can  they  be  measured  so  as  to  make 
a  prediction  or  a  statement  of  the  probability  of  success  in  a 
particular  job?  Summarizing  these  questions,  we  state  the  psy- 
chological problem  as  follows :  To  determine  what  mental  abili- 
ties and  traits  are  of  practicable  use  as  criteria  for  differentiating 
graduate  engineers  and  predicting  their  success  in  six  different 
but  closely  related  kinds  of  work.  This  problem  includes  the 
evaluation  of  old  methods  and  means  and  the  devising  of  new- 
methods  and  means  for  determining  and  measuring  these  mental 
abilities  and  traits.  It  is  not  to  be  expected  that  we  should  answer 
all  the  questions  stated  above,  which  have  already  taken  years 
of  work  by  the  best  psychologists,  and  will  require  much  more 
work;  but  these  questions  should  be  recognized  as  pertinent  in 
the  assumptions  and  conclusions  of  this  study. 

This  being  a  specific  study  with  a  definite  problem  in  applied 
psychology,  it  does  not  include  in  its  purpose  and  scope  any  his- 
torical or  theoretical  treatment  of  many  general  psychological 
problems  which  may  be  touched  upon.  Very  few  studies  that 
have  been  published  deal  directly  with  this  kind  of  problem. 
However,  at  the  end  of  this  report  is  appended  a  bibliography  or 
list  of  studies  which  bear  more  or  less  specifically  upon  a  prob- 
lem of  this  nature. 
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PART  II.     THE  LINES  OF  INVESTIGATION 
AND  THE  RESULTS 

III.     Occupational  Descriptions  and  Analyses 

The  first  important  step  in  all  personnel  selection  and  place- 
ment is  to  secure  adequate  descriptions  and  analyses  of  the  jobs 
to  be  filled.  Then  from  these  there  must  be  prepared  specifica- 
tions for  the  persons  who  will  most  nearly  meet  these  require- 
ments. The  specification  is  an  interpretation  of  the  work  done 
on  the  job,  in  terms  of  capacities,  skill,  knowledge,  and  traits  re- 
quired to  do  the  work.  In  practice  the  degree  of  specificity  to 
which  these  descriptions  are  carried  varies  considerably  from 
firm  to  firm  and  with  different  lines  of  work.  In  general  dis- 
cussions it  is  quite  common  to  refer  to  the  various  vocations, 
which  differentiate  only  the  general  lines  of  work  typified  by  the 
various  professions  and  trades.  Often  in  employment  and  place- 
ment work,  differentiations  are  made  only  into  occupations,  which 
are  understood  to  be  the  specialties  in  the  trades,  such  as  punch 
press  hand,  lathe  hand,  die  sinker,  armature  winder,  etc.  Strictly 
speaking,  the  job  description  is  reserved  for  a  description  of  a 
particular  operation  on  a  particular  piece  of  material  or  appara- 
tus by  means  of  a  particular  tool  or  machine.  In  the  Westing- 
house  Electric  and  Manufacturing  Company  these  distinctions 
are  made  in  the  use  of  these  terms  by  the  Occupations  and  Rate 
Committee  and  by  those  in  the  Employment  Department. 

In  the  Westinghouse  Electric  and  Manufacturing  Company 
the  employment  and  placement  of  the  engineers  is  done  by  the 
Educational  Department.  Each  spring  this  department  selects 
between  two  and  three  hundred  engineers  from  the  senior  classes 
of  the  technical  colleges;  and  after  giving  them  a  year  of  grad- 
uate training  in  the  Educational  Department  at  the  central  plant 
in  East  Pittsburgh,  it  places  these  men  in  the  various  lines  of 
work  in  the  organization.  Before  it  was  possible  to  choose  any 
certain  engineers  as  better  fitted  for  certain  lines  of  work,  it  was 
necessary  to  know  what  was  expected  of  these  engineers  in  each 
line  of  work.  As  there  were  not  on  record  any  descriptions  or 
specifications  of  the  work  done  by  the  engineers  in  the  various 
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departments,  this  information  had  to  be  gotten  first.  The  descrip- 
tions of  the  lines  of  work  were  obtained  through  the  Educational 
Department. 

After  conferences  with  some  of  the  executives  who  could  tell 
most  about  the  various  lines  of  work,  a  description  and  outline  of 
the  nature  of  the  occupational  description  desired  was  given  to  the 
Educational  Department.  Under  the  supervision  of  the  Educa- 
tional Department  the  occupational  descriptions  were  prepared 
by  the  heads  of  the  various  sections  in  each  of  the  departments. 
These  were  elaborated  further  by  means  of  conferences  with 
various  executives  in  which  questions  were  asked  to  clear  up 
obscure  details.  Finally,  in  this  manner,  a  conception  of  each 
line  of  work  was  obtained,  which  was  detailed  enough  for  work- 
ing intelligently  on  the  problem  of  selection  and  placement. 

It  would  be  out  of  place  here  to  give  an  exhaustive  description 
of  each  job  or  line  of  work  in  each  section  of  each  department. 
Moreover,  such  detailed  descriptions  are  not  necessary  for  un- 
derstanding the  problem  that  is  before  us.  The  complete  occu- 
pational descriptions  are  on  file  with  the  Division  of  Applied  Psy- 
chology, Carnegie  Institute  of  Technology,  and  are  available  for 
those  who  wish  to  examine  them.  It  is  sufficient  here  to  have  a 
description  of  the  main  classes  of  engineering  work.  In  actual 
work  upon  the  problem,  only  four  classes  of  engineers  are  con- 
sidered, namely :     ( i )   research  workers  and  design  engineers ; 

(2)  general  or  all  other  engineers  in  the  engineering  department; 

(3)  operating,  service,  and  works  management  engineers;  (4) 
sales  engineers.  This  was  found  practicable  because  nearly  all 
the  engineers  were  graduates  of  technical  colleges  and  had  re- 
ceived essentially  the  same  technical  training.  Moreover,  this 
process  of  being  admitted  to  the  technical  colleges  and  of  being 
graduated  from  them,  had  been  highly  selective  so  that  any  dif- 
ferentiation among  the  men  was  possible  only  by  comparatively 
fine  discrimination.  In  fact,  the  one  real  differentiation  that  was 
consistently  kept  in  mind  in  all  this  study  was  that  between  the 
engineers  going  into  the  sales  department  and  all  other  engineers. 
This  might  be  qualified  or  rather  more  definitely  stated  by  ex- 
plaining that  the  design  engineers  were  also  differentiated  from 
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the  other  engineers,  and  being  considered  as  representing  the 
purest  type  of  engineer,  they  were  also  considered  as  at  the 
opposite  extreme  from  the  sales  engineer  when  the  differences 
were  looked  at  as  quantitative  or  linear  in  nature.  The  descrip- 
tions of  these  lines  of  work  are  given  with  particular  attention  to 
those  lines  considered  to  be  most  clearly  differentiated.  Com- 
bined with  these  descriptions  are  given  the  specifications  for  the 
kind  of  men  considered  best  fitted  for  each  particular  line  of 
work. 

i.  Functional  Classification  of  Engineers 
Within  the  organization,  the  engineers  are  classified  according 
to  the  department  and  according  to  the  class  of  product  worked 
upon.  Since  the  specifications  for  the  kind  of  engineer  needed 
depends  upon  the  kind  of  work  done,  we  must  classify  the  en- 
gineeers  according  to  the  qualifications  required  to  do  the  par- 
ticular kinds  of  work:  (a)  In  all  the  specialized  engineering 
departments  which  are  responsible  for  the  engineering  work  on 
different  particular  lines  of  electrical  apparatus  and  machinery, 
thus  excepting  the  General  Engineering  Department  and  the 
Testing  Department,  there  must  be  design  engineering.  This 
calls  for  engineering  of  the  purest  and  highest  type.  With  this 
type  of  engineering  is  also  classed  the  research  worker  in  the 
Research  Department,  (b)  All  the  other  engineers  of  the  en- 
gineering departments  are  to  be  considered  as  doing  a  second 
type  of  work.  This  work  is  engineering ;  but  it  is  not  to  be  con- 
sidered as  calling  for  as  great  mathematical  and  technical  ability 
and  as  much  originality  in  that  line  as  is  needed  by  the  design 
engineer,  (c)  The  third  class  includes  the  engineers  in  Factory 
Management,  the  Service  Department,  and  Operating.  The 
work  of  these  engineers  brings  them,  to  a  larger  extent,  in  con- 
tact with  other  people  outside  the  organization  and  also  with 
those  inside  the  organization.  Their  work  requires  more  exec- 
utive ability  than  the  work  done  by  the  other  groups  of  engin- 
eers, (d)  The  fourth  class  is  that  composed  of  Sales  Engineers. 
These  men  are  primarily  engineers  as  well  as  salesmen ;  for  to 
get  and  keep  the  good-will  and  confidence  of  the  customers,  they 
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must  act  as  consulting  engineers  and  give  reliable  technical  advice 
to  the  customer  or  prospective  customer.  However,  the  success- 
ful engineer  doing  the  work  in  the  sales  department  must  have 
a  commercial  sense  and  an  ability  to  meet  and  successfully  nego- 
tiate with  other  people,  that  differentiates  him  from  the  other 
engineers. 

IV.     Criteria  of  Success  of  Engineers 

Closely  allied  and  of  equal  importance  with  a  definite  know- 
ledge of  the  jobs  to  be  filled,  are  criteria  of  success  of  men  in 
those  jobs.  Methods  of  selection  and  placement  cannot  be  de- 
veloped or  improved  upon  unless  it  is  possible  to  know  when  bet- 
ter results  are  obtained.  The  best  criteria  are  objective  measures, 
such  as  records  of  production;  but  it  is  seldom  possible  to 
obtain  facts  on  such  objective  measures  that  are  comparable. 
The  work  of  sales  engineers  and  of  design  engineers  is  such 
that  it  cannot  be  measured  by  the  product  finished.  Although 
records  of  sales  may  be  kept,  many  other  factors  enter  into  the 
value  of  a  sales  engineer  to  the  company.  The  professional 
consulting  service  which  the  engineer  gives  to  the  customer,  and 
the  good-will  secured  in  return,  cannot  be  measured  in  that  way. 
Also,  a  design  engineer  may  spend  all  his  time  for  a  year  or 
more  on  one  machine  or  piece  of  apparatus;  but  if  he  gives  the 
piece  of  apparatus  an  original  design  or  makes  it  in  the  nature  of 
an  invention,  his  work  is  scarcely  measurable.  However,  it  is 
necessary  to  make  use  of  the  best  criteria  available. 

The  year  spent  in  the  Educational  Department  can  be  consid- 
ered as  one  phase  of  the  engineer's  work.  Therefore,  grades  and 
ratings  on  their  success  there  are  some  criteria.  Furthermore, 
the  executives  and  instructors  of  that  department  come  to  know 
intimately  all  the  student  engineers  during  the  year  of  training. 
To  give  some  definite  data  against  which  tests  and  other  ratings 
might  be  checked,  the  Educational  Department  was  asked  to 
rank  the  student  engineers  in  the  order  of  their  general  intel- 
ligence. General  intelligence  was  defined  for  them  as  "the  ability 
to  analyze  a  problem,  grasp  the  point,  and  deal  with  a  new  situa- 
tion."   Two  men  who  both  supervise  the  work  of  the  engineers 
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and  instruct  them,  worked  together  as  a  committee  to  rate  the 
student  engineers.  They  first  classified  the  men  into  five  groups 
according  to  their  general  intelligence,  as  defined.  Then  they 
ranked  the  men  in  each  group  in  the  order  of  their  estimated 
intelligence.  By  combining  the  groups,  with  the  highest  rated 
group  first,  a  ranking  of  all  the  engineers  was  secured. 

Further  checks  on  the  reliability  of  ratings,  test  scores,  and 
these  grades  were  possible  by  intercorrelations.  Although  none  of 
these  were  absolute  measures  of  success,  it  was  reasonable  to 
assume  that  high  intercorrelations  between  different  kinds  of 
measures,  such  as  between  grades  and  tests,  or  between  estimates 
and  grades  would  show  that  there  was  a  certain  consistency  in 
the  measures  that  correlated  highly  with  other  measures,  and 
that  there  was  no  cause  to  believe  that  they  were  unreliable. 

V.     Evaluation  of  Technical  School  Grades 

The  representatives  of  the  Company,  in  selecting  engineers 
graduating  from  the  technical  schools,  did  not  consider  primarily 
the  grades  received  by  the  engineers  in  their  college  courses.  They 
interviewed  the  seniors  for  twenty  or  thirty  minutes,  and  then 
later  inquired  about  the  man's  grades  in  general.  However,  the 
grades  were  not  considered  as  an  absolute  index  of  a  man.  Lead- 
ership and  participation  in  extra-academic  activities  were  con- 
sidered as  important.  It  is  not  always  easy  to  obtain  the  grades 
in  such  a  way  that  extensive  and  consistent  use  can  be  made  of 
them.  I  thought  that  probably  they  would  show  something  of  the 
technical  training  and  ability  of  the  men,  but  that  many  qualities 
making  for  success  in  engineering  and  salesmanship  were  prob- 
ably not  to  be  inferred  from  such  grades.  However,  a  scientific 
investigation  of  this  sort  would  not  ignore  them.  Moreover,  a 
study  of  these  grades  might  give  some  light  on  the  significance 
of  grades  given  in  the  Company's  training  courses.  Accordingly, 
I  made  a  study  of  the  grades  which  graduate  engineers  received 
in  the  technical  schools.  Many  difficulties  were  encountered  in 
attempting  to  deal  with  technical  school  grades  statistically  and 
to  derive  reliable  conclusions. 

Transmutation  and  Tabulation  of  Technical  School  Grades. — 
A  letter  explaining  the  nature  of  this  study  and  the  purpose  in 
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requesting  the  grades  was  sent  to  over  sixty  different  technical 
schools  to  secure  the  grades  of   116  students.     Transcripts  of 
grades  for  94  students  were  received.     No  two  schools  used  the 
same  form  on  which  to  record  the  grades ;  and  there  were  at  least 
seven  distinctly  different  systems  of  grading  with  several  varia- 
tions of  each  system.    The  grades  of  men  from  different  schools 
had  to  be  made  comparable.     There  were  not  enough  men  from 
any  one  school  to  make  it  possible  to  transmute  their  grades  into 
standard  deviations,  so  another  method  was  adopted.    The  range 
of  possible  grades  between  the  just  passing  grade  and  the  highest 
possible  grade,  was  learned  either  from  the  college  catalogue  or 
from  the  key  on  the  transcript  of  grades.     The  median  point  on 
that  scale  was  estimated  and  called  the  average  grade.    All  such 
average  grades  were  given  a  marking  or  value  of  2.     Grades 
above  average  were  given  a  value  of  3 ;  and  grades  below  average 
were  given  a  value  of  1.     Grades  below  passing  were  given  a 
value  of  o;  and  exceptionally  high  grades  in  a  subject,  shown 
to  be  reliable  by  more  than  one  highest  marking  on  that  subject, 
were  given  a  value  of  4.     The  method  of  transmutation  of  the 
grades  can  be  shown  by  the  following  table  which  gives  some 
comparable  values. 

Table  I. 
Transmutation  of  Technical    School   Grades   into  Common  Values. 


Common    Values    into 

which  Grades  are 

Transmuted. 


System  using  per  cents.. 

-69 

1 

70-79 

System  using  letters 

E 

D 

System   using  passing 
marks    

F 
1 

P— 

80-89 


System  using  descriptive 
terms 


Fail 


Poor 


Good 


90-97 


98-100 


B 


P+ 


All 

P+ 


Superior        Excellent 


System  using  merits 

Fail 

Pass    with 
low  mark 

Pass 

. . 

Merit             Honor 

System  using  points 

0-.9               1. 0-1.9 

2.0-2.9 

3.0-3.7             3-8-4-0 

System  using  class  rank. . 

10%  of  class 20%  of  class _to%  of  class 

20%  of  class  10%  of  class 

io  BRUCE  V.  MOORE 

In  transmuting  grades  from  institutions  with  high  standards 
and  strict  grading  systems,  the  evaluations  were  made  higher 
than  with  institutions  known  to  be  below  the  average  standing  in 
the  requirements  it  made  of  its  students.  Altogether,  the  trans- 
mutation was  coarse  as  a  statistical  method;  but  it  was  the  best 
that  could  be  done  with  the  data  available;  and  it  is  considered 
sufficiently  reliable  to  give  significant  indications. 

All  the  subjects  studied  by  the  engineers  were  classified  under 
five  headings  or  general  subjects,  namely, — Languages,  including 
English  and  foreign  languages;  Mathematics  and  Science,  in- 
cluding mechanics,  physics,  chemistry,  and  all  special  phases  of 
these  exact  and  theoretical  sciences;  Shop;  Engineering,  includ- 
ing all  kinds  of  engineering,  but  chiefly  electrical  and  some 
mechanical;  and  all  Academic  subjects,  which  is,  in  the  case  of 
engineers,  only  history  and  economics. 

Comparison  of  Technical  School  Grades  with  Other  Criteria. — 
The  first  question  to  raise  about  the  grades  is  in  regard  to  their 
relation  to  success  in  industry  or  practical  work.  These  en- 
gineers had  not  yet  been  in  full-time  work,  but  their  year  of 
work  in  the  Educational  Department  and  in  the  shop  under  the 
direction  of  that  department,  could  be  considered  as  one  form  of 
practical  work  in  which  success  is  to  be  determined.  The  esti- 
mates made  by  the  committee  of  the  Educational  Department, 
explained  under  the  title,  "Criteria  of  Success,"  might  be  con- 
sidered as  one  rating  of  success  in  the  Educational  Department. 
Grades  given  in  the  Educational  Department  classes,  and  grades 
given  by  the  foremen  on  the  shop  work,  can  be  considered  as 
another  check.  Ratings  on  various  qualities,  explained  in  a  sec- 
tion later,  could  be  considered  as  the  success  of  the  men  in  im- 
pressing the  foremen  with  their  ability,  and  might  be  related  to 
school  grades.  Also,  the  correlation  between  technical  school 
grades  and  the  ratings  of  the  men  made  by  the  representatives  of 
the  Educational  Department  at  the  time  of  the  employment  inter- 
view, might  be  some  measure  of  the  extent  to  which  the  two 
measure  the  same  qualities.  Finally,  a  positive  correlation  be- 
tween mental  alertness  or  general  intelligence  test  and  college 
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grades  would  be  at  least  some  evidence  that  school  grades  were 
some  indication  of  what  is  measured  by  tests  as  general  intel- 
ligence or  mental  ability.  The  correlations  between  technical 
school  grades  and  the  various  other  measures  or  criteria  are  as 
shown  in  Table  II. 

Table  II. 

Correlations  of  Technical  School  Grades  with  Other  Measures. 

No.     Correlation 
of  cases       (r) 
Technical    school    grades     with     estimated     intelligence, 

committee's   rating    88  .0 

Technical    school    grades    with    educational    department 

class   grades    60  .0 

Technical  school  grades  with  foremen's  shop  grades...  62  .0 

Technical  school  grades  with  foremen's  ratings  on  traits  85  .0 
Technical  school  grades  with  ratings  at   interview   for 

employment     21  +.22 

Technical  school  grades  with  Bureau  Test  VI 56  +-37 

Grades  in  languages  with  foremen's  ratings  on  traits ...  59  .0 

Grades  in  mathematics  with  foremen's  ratings  on  traits . .  72  .0 

Grades  in  shop  with  foremen's  ratings  on  traits... 31  .0 

Grades  in  engineering  with  foremen's  ratings  on  traits..  66  .0 
Grades  in  academic  subjects  with  foremen's  ratings  on 

traits    44  0 

Grades  in  languages  with  Bureau  Test  VI 43  .0 

Grades  in  mathematics  with  Bureau  Test  VI 54  +-34 

Grades  in  shop  with  Bureau  Test  VI 24  -j— 67 

Grades  in  engineering  with  Bureau  Test  VI 53  +.26 

Grades  in  academic  subjects  with  Bureau  Test  VI 35  .0 

The  correlation  coefficients  between  technical  school  grades 
and  other  criteria,  particularly  the  Company's  Educational  De- 
partment class  grades,  shop  grades,  and  shop  ratings,  might  be 
taken  to  indicate  that  the  technical  school  grades  are  unreliable. 
However,  I  do  not  believe  that  this  conclusion  is  justified. 
Rather,  I  believe  that  the  shop  and  class  grades,  and  the  ratings 
are  unreliable,  or  are  largely  the  cause  for  the  lack  of  correlation. 
The  cause  for  this  belief  will  be  shown  later  in  this  report. 

The  composite  of  technical  school  grades  shows  a  significant 
correlation  with  intelligence  test  scores.  Doubtless  the  correla- 
tions would  be  higher  if  the  grades  were  not  from  so  many  dif- 
ferent schools  with  different  standards.  I  believe  that  in  any 
particular  school,  an  average  or  composite  of  the  grades  received 
by  each  student  would  rank  the  students  in  general  ability  as  well 
as  any  measure;  but  the  standards  of  work  and  the  methods  of 
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grading  are  so  incomparable  in  different  schools  that  they  can 
not  be  used  by  the  employer  to  compare  men  from  different 
schools.  A  standardized  psychological  test  would  show  the  same 
thing  in  much  more  reliable  and  usable  form.  That  is,  a  general 
intelligence  test  given  to  a  group  of  engineers  selected  from 
sixty  different  technical  schools  would  give  a  more  reliable  rank- 
ing of  the  men  according  to  their  ability  than  a  comparison  of 
grades  from  these  different  schools;  and  the  ranking  could  be 
obtained  much  easier  by  a  test  than  by  an  attempt  to  transmute 
various  systems  of  grades  into  a  common  scale. 

The  intercorrelations  of  the  grades  in  the  various  subjects  is 
shown  in  Table  III.  As  might  be  expected,  the  correlation  be- 
tween closely  related  subjects,  such  as  Mathematics  and  Engin- 
eering, (r  =  +.57)  is  greater  than  between  what  are  usually 
considered  less  closely  related  subjects,  such  as  Academic  Sub- 
jects and  Shop  (r  =  -(-.30).  There  might  be  some  a  priori 
question  about  the  correlations  between  Shop  and  Mathematics 
or  between  Shop  and  Engineering,  but  the  correlations  are  shown 
to  be  comparatively  high,  being  (r  =  +.61 )  (r  =  -f--42)  respect- 
ively. Shop  grades  correlate  higher  with  Bureau  Test  VI  scores 
than  any  other  technical  school  subjects  do ;  but  the  number  of  cases 
is  so  small  that  the  large  probable  error  makes  the  correlation  co- 
efficient highly  unreliable.  However,  it  will  be  seen  later  that  the 
design  engineers,  who  are  picked  men  and  are  supposed  to  be  the 
most  capable  men  in  pure  mathematics  and  engineering,  have  the 
highest  grades  in  Shop,  and  are  differentiated  more  widely  from 
the  other  engineers  by  the  average  of  their  grades  in  Shop.  It 
seems  that  the  grades  that  a  man  receives  in  his  shop  work  in 
the  technical  schools  is  significant  of  his  special  engineering  apti- 
tude, and  even  of  his  general  mental  capacity. 

For  differentiating  the  engineers  for  the  different  lines  of 
work  which  they  are  to  do,  I  thought  that  grades  in  different 
subjects  might  be  significant.  The  average  grade  in  each  main 
subject  for  each  class  of  engineers  was  computed,  and  curves 
plotted  to  show  the  relation  of  one  group  of  engineers  to  the 
others.     (See  Diagram  1.)     The  design  engineers  are  above  the 
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Table  III. 

Intercorrelations   of    Technical    School    Subjects. 
Subjects  of  cases  (j) 

Number      Correjation 

Languages    with    mathematics T]  +-38 

Languages  with  shop    35  +-37 

Languages  with  engineering   69  +-55 

Languages  with  academic  subjects   53  +-3° 

Mathematics  with  shop  43  +.61 

Mathematics  with  engineering   85  +-57 

Mathematics  with  academic  subjects   57  -K42 

Shop  with  engineering   63  +42 

Shop  with  academic  subjects    21  +-3° 

Engineering  with  academic  subjects   54  +-35 

other  engineers  in  every  subject,  indicating  that  they  were  su- 
perior students  in  the  technical  schools.  The  curves  of  the  gen- 
eral engineers  and  of  the  operating,  service,  and  works  manage- 
ment engineers  run  comparatively  close  together,  the  average  of 
one  group  in  a  subject  rising  or  falling  with  the  average  of  the 
other  group  of  engineers.  The  average  of  the  sales  engineers 
varies  inversely  with  the  averages  of  the  general  and  the  oper- 
ating, works,  and  service  engineers.  The  significant  feature  of 
the  averages,  however,  is  that  the  design  engineers  are  very 
much  higher  in  all  subjects,  except  the  academic  subjects,  com- 
posed chiefly  of  economics,  while  in  this  subject  the  sales  en- 
gineers have  almost  as  high  an  average,  although  the  general  or 
all-around  ability  of  the  design  engineers  is  evidently  higher. 
The  man  who  has  been  selected  for  other  reasons  as  promising 
of  becoming  a  good  sales  engineer,  is  the  one  who  had  special 
ability  in  the  social  sciences. 

VI.     Evaluation  of  Grades  and  Ratings  Given  During 

Training  in  the  Educational  Department  of 

the  Industrial  Firm 

As  soon  as  the  engineers,  usually  just  graduated  from  the  tech- 
nical school,  enter  the  employ  of  the  Company,  they  begin  the 
year  of  training  in  the  Educational  Department.  Practically 
all  the  engineers  spend  the  first  two  months  taking  the  same 
courses  and  doing  the  same  work.  In  addition  to  class  work, 
they  work  about  a  month  in  one  department  of  the  shop  and 
then  pass  on  to  another  kind  of  work  so  as  to  become  familiar 
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with  as  many  phases  of  the  manufacturing  process  as  possible. 
To  make  the  absorption  of  the  men  into  the  shop  possible  and 
efficient,  the  men  have  to  be  divided  into  many  groups  and  sent 
into  many  different  sections,  but  the  work  for  all  is  very  similar 
at  first.  At  the  end  of  two  months,  however,  there  is  begun  the 
work  of  segregating  the  men  into  groups  according  to  the  line 
of  work  that  they  will  do  permanently.  That  is,  the  sales  en- 
gineers and  the  design  engineers,  works  management  engineers, 
general  engineers,  etc.,  are  picked  out.  This  process  of  trans- 
ferring the  men  from  one  department  to  another  and  of  deter- 
mining to  what  line  of  work  each  will  be  eventually  assigned,  con- 
tinues until  the  end  of  the  sixth  month.  Beginning  with  the  sixth 
month  a  Sales  School  and  a  Design  School  are  started;  and  the 
engineers  begin  specialized  training  for  their  future  work. 

From  the  time  that  the  engineers  enter  the  Educational  De- 
partment, extensive  records  of  their  work  are  kept.  Most  of  the 
men  are  first  assigned  to  industrial  motor  winding  and  are  graded 
on  that  work,  both  in  the  classes  and  in  the  actual  work  in 
the  shop.  In  this  work,  the  correlation  between  the  shop  grades 
and  the  class  grades  was  r  =  +.19,  for  76  cases.  The  grades 
for  any  one  group  of  engineers  plotted  separately  showed  no 
different  correlations.  This  low  correlation  shows  the  disagree- 
ment of  the  impressions  received  by  different  people,  and  the  un- 
reliability of  opinions  concerning  traits  in  people,  particularly 
when  the  judgment  must  be  made  only  after  a  short  period  of 
observation  of  the  subject's  work.  The  curves  in  Diagram  2, 
show  that  the  design  engineers  receive  the  highest  grades  in 
the  shops.  This  agrees  with  the  fact  that  they  received  the  high- 
est grades  in  Shop  in  the  technical  school  grades. 

In  addition  to  the  grades  in  the  different  subjects  or  kinds  of 
work,  there  come  from  the  foremen  over  each  department,  ratings 
on  various  traits  of  the  engineers  working  temporarily  under  these 
foremen.  These  ratings  are  on  ten  different  traits  or  characteris- 
tics; and  they  are  made  in  terms  of  A,  B,  and  C.  The  names  of 
the  ten  traits  are  reliability,  industry,  initiative,  tact,  attitude, 
analytical    ability,    aptitude,    enthusiasm,   personality,   decision. 
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Since  the  men  are  transferred  each  month  and  rated  by  different 
foremen,  several  ratings  in  each  trait  are  recorded.  By  combin- 
ing these  ratings  a  more  discriminating  rating  on  each  man  of  the 
group  being  studied  was  obtained.  That  is,  the  values  A,  B,  and 
C  were  transmuted  into  the  values  2,  i,  and  o,  respectively;  and  by 
combining  and  averaging  these  ratings,  fractional  values  be- 
tween these  integers  were  obtained,  such  as  0.8  or  1.3  or  1.7  as 
a  man's  average  rating.  Correlations  of  these  shop  ratings  with 
other  data  on  the  men,  are  given  in  Table  IV. 

Table  IV. 

Correlations  of   Shop  Ratings  with   Other   Criteria. 

Number      Correlation 
Subjects  of  cases  (r) 

Shop  ratings  with  estimated  intelligence,  committee's 
rating 95  .0 

Shop  ratings  on  analytical  ability  with  estimated  in- 
telligence,  committee's   rating 94  .0 

Shop  ratings  with  Bureau  Test  VI  scores 58  .0 

Shop  ratings  on  analytical  ability  with  Bureau  Test  ■ 
VI   scores    56  .0 

Shop  ratings  with  ratings  at  interview  for  employ- 
ment      27  .0 

Shop  ratings  on  analytical  ability  with  ratings  on  in- 
telligence at  interview  for  employment 26  .0 

Shop  ratings  on  personality  with  ratings  on  personal 
qualities  at  employment   interview 27  +-25 

Shop  grades  with  shop  ratings  74  .0 

From  these  correlations  we  conclude  that  the  ratings  made 
by  the  foremen  on  the  men  working  under  them  only  one  month, 
are  very  unreliable,  and  really  indicate  nothing.  Doubtless,  this 
unreliability  cannot  all  be  charged  to  the  inability  of  the  fore- 
men to  judge  the  men  on  some  of  these  traits;  but  the  method 
of  rating,  of  recording  these  ratings,  and  of  combining  ratings 
by  different  foremen,  makes  them  incomparable  and  unreliable. 
In  the  shop  and  the  Educational  Department,  the  interpretations 
of  analytical  ability  and  of  aptitude  are  practically  the  same  as 
that  which  is  called  general  intelligence  by  the  psychologists.  The 
ratings  on  analytical  ability  were  correlated  with  estimated  intel- 
ligence and  with  test  scores  in  the  hope  that  a  significant  positive 
correlation  would  be  found,  but  there  was  none.  One  exception 
to  the  lack  of  positive  correlation  is  the  small  correlation  be- 
tween personality  as  judged  by  foremen  and  personal  qualities 
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as  judged  by  the  representatives  of  the  company  who  rated  the 
men  at  the  interview  for  employment.  It  seems  reasonable  to 
expect  such  a  positive  correlation ;  for  it  is  doubtless  the  general 
personality  which  influences  both  the  interviewer's  and  the  fore- 
man's rating  on  a  man  in  most  of  the  traits,  in  much  the  same 
way. 

Table  V. 

Some  Intercorrelations  of  Ratings  by  Foremen  on  Traits  of  Engineers. 

Number  Correlation 

of  cases  (r) 

Reliability  with  industry    117  +.66 

Initiative  with  tact   113  +.66 

Initiative  with  enthusiasm   107  +42 

Analytical  ability  with  aptitude   102  -\--72 

Analytical  ability  with  personality   104  +.26 

Enthusiasm  with  personality    106  +-57 

Correlations  between  the  ratings  on  some  of  the  traits  were 
computed.  The  highest  correlation  found  is  between  analytical 
ability  and  aptitude,  -f-  .72,  which  seems  a  priori  reasonable.  The 
next  highest  correlation  is  (r  =  +.66)  between  reliability  and  in- 
dustry, which  might  be  expected;  but  there  is  the  same  correla- 
tion between  tact  and  initiative,  two  traits  which  a  priori  we 
should  judge  would  tend  to  be  found  in  somewhat  opposite 
types  of  individuals.  The  lowest  correlation  is  between  analytical 
ability  and  personality,  which  does  not  seem  unreasonable. 

To  discover  whether  the  ratings  on  the  ten  different  traits 
could  be  of  any  value  in  differentiating  the  engineers,  the  average 
of  the  ratings  on  each  trait  for  each  group  was  computed.  These 
averages  are  shown  in  Diagram  2.  The  curves  tend  to  follow 
each  other  very  closely.  Also,  it  was  found  in  most  of  the  rat- 
ings that  if  a  man  was  rated  high  in  one  trait,  he  was  rated  com- 
paratively high  in  another.  Personality  and  enthusiasm  differ- 
entiate the  groups  most  widely.  Frequency  column  diagrams, 
Diagrams  3,  4,  5,  and  6,  show  that  the  ratings  in  these  two 
traits  and  in  tact  and  initiative,  tended  to  separate  the  sales  en- 
gineers, and  the  work  management  engineers  from  the  other 
engineers.  By  combining  the  ratings  in  these  four  traits  we  get 
a  very  definite  differentiation  of  the  four  groups  of  engineers. 
See  Diagram  2.    It  seems  evident  that  people  on  coming  in  con- 
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tact  with  the  different  groups  of  engineers  sense  a  difference  in 
the  personal  traits  of  these  men ;  but  definite  analysis  and  measure- 
ment of  this  difference  is  difficult  in  any  particular  individual. 

VII.  A  Study  of  Rating  Scales 
For  differentiating  sales  engineers  from  other  engineers,  it  is 
probably  true  that  the  personal  qualities  aside  from  intelligence 
are  more  important.  This  has  been  recognized  in  the  attempt 
to  estimate  these  qualities  as  a  whole  without  a  rating  scale  when 
a  man  was  being  considered  for  any  particular  work.  There  are 
at  present  no  psychological  tests  for  measuring  these  qualities, 
but  there  are  refined  methods  of  making  estimates  or  ratings. 
The  rating  scale  is  a  tool  or  means  for  carrying  out  such  a  more 
accurate  method.  Its  value  as  a  measure  of  various  traits  de- 
pends upon  the  standards  which  it  sets  up  as  measuring  rods,  and 
its  accuracy  depends  upon  the  definiteness  of  these  standards. 

1.  Principles  of  a  Rating  Scale 
Employers  have  always  tried  to  estimate  the  value  of  a  man  as 
a  whole,  considering  his  qualities  altogether  without  the  use  of  a 
rating  scale.  Great  differences  in  the  estimates  made  by  different 
persons  on  the  same  man  have  shown  this  to  be  an  uncertain 
method.  The  man's  qualities  must  be  separated  and  rated  by 
themselves;  for  if  they  are  not,  the  estimator  is  likely  to  let  his 
prejudices  consider  only  one  prominent  quality  and  let  them  bias 
his  judgment  so  as  to  neglect  all  other  qualities.  For  example, 
a  foreman  or  supervisor  may  have  an  intense  dislike  for  one  of 
his  workers  because  the  worker  has  a  disfigured  face,  and  un- 
consciously the  foreman  overlooks  the  man's  other  good  quali- 
ties. However,  when  the  rating  scale  calls  for  a  consideration 
of  these  qualities  separately,  he  is  forced  to  realize  that  the 
worker  may  be  intelligent,  industrious,  skillful,  and  cooperative. 
After  the  foreman  is  led  to  face  the  facts  in  this  way,  he  must 
admit  that,  after  all,  the  man  has  good  points  that  are  valuable  in 
a  workman.  On  the  other  hand,  the  foreman  may  have  to  admit 
that  he  has  been  overlooking  another  workman's  laziness  and 
lack  of  skill  because  this  man  was  always  good-natured  and  so- 
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ciable.  Moreover,  the  fact  that  these  ratings  must  be  signed  and 
given  in  black  and  white  to  a  superior  who  can  check  them, 
leads  the  foreman  to  consider  sources  of  inefficiency  in  his  work- 
ers. It  also  makes  him  more  careful  and  fair-minded  in  his  judg- 
ments of  his  men. 

We  can  then  lay  down  as  the  first  principle  of  the  rating  scale 
that:  (i)  Instead  of  trying  to  estimate  a  person's  ability  as  a 
whole  for  doing  any  particular  work,  this  ability  is  analyzed  into 
component  essential  abilities  or  traits,  and  each  trait  is  rated  inde- 
pendently of  the  others.  The  traits  selected  should  be  the  most 
important  for  success  in  the  particular  situation.  It  is  much 
better  to  have  a  few  traits  that  are  really  essential  and  critical 
for  a  worker's  success  than  to  have  ratings  on  many  traits,  some 
of  which  are  relatively  unimportant.  That  is,  it  is  much  better 
to  concentrate  on  a  few  important  qualities  and  have  ratings 
on  these  made  and  really  used  than  to  have  a  system  so  elaborate 
that  it  breaks  down  from  its  own  size  and  complexity.  The  traits 
included  in  the  rating  scales  discussed  here  were  selected  for 
rating  men  under  certain  conditions  for  a  definite  purpose.  Any- 
one using  this  scale  would  have  to  consider  these  traits  for  their 
particular  purpose  in  the  light  of  the  principles  stated  here. 

The  reliability  of  a  rating  scale  as  a  measure  of  various  traits 
depends  upon  the  standards,  particularly  the  definiteness  of  the 
standards  which  it  sets  up  as  measuring  rods.  Therefore,  other 
principles  for  making  an  accurate  rating  scale  are  to  be  observed ; 
and  the  next  principle  is:  (2)  the  different  traits  determined 
upon  must  be  really  different  and  as  distinct  from  each  other  as 
possible.  They  must  be  supplementary  to  each  other  with  the 
minimum  of  indefiniteness  and  overlapping. 

(3)  For  rating  a  person  in  any  trait,  the  person  doing  the 
rating  must  be  acquainted  with  the  one  to  be  rated,  and  have 
more  or  less  dependable  facts  for  making  a  decision.  When  an 
employee  asks  for  a  promotion  or  transfer,  the  employment 
manager  cannot  rate  the  employe  in  cooperation  or  in  industry; 
but  the  worker's  previous  foreman  probably  could.  On  the  other 
hand  the  employment  manager  might  be  a  better  judge  of  the 


PERSONNEL  SELECTION  OF  GRADUATE  ENGINEERS         23 

applicant's  appearance  and  manner  in  considering  him  for  trans- 
fer to  the  office  or  some  place  where  he  would  meet  the  public. 

(4)  The  traits  must  be  as  sharply  defined  as  possible  so  that 
different  people  doing  the  rating  will  rate  the  same  trait.  In 
naming  and  defining  traits,  use  terms  which  are  more  or  less 
common  and  have,  as  nearly  as  possible,  universally  the  same 
meanings  accepted  by  everybody  and  the  minimum  chances  of 
ambiguity  and  misunderstanding. 

(5)  For  rating  an  individual  in  a  trait,  the  basis  of  comparison 
used  as  a  scale  should  be  as  concrete  and  as  familiar  as  possible. 
To  provide  this  condition,  the  person  to  be  rated  is  compared  in 
regard  to  a  particular  trait  with  other  well-known  persons  who 
differ  in  the  extent  to  which  they  have  that  particular  trait.  That 
is,  a  foreman,  in  rating  a  man,  should  compare  him  with  men 
who  are  doing  the  same  kind  of  work  under  similar  circum- 
stances ;  and  the  foreman  must  be  well  acquainted  with  the  traits 
and  qualifications  of  the  men. 

(6)  Where  more  than  one  individual  is  to  be  rated  in  more 
than  one  trait,  more  comparable  results  are  obtained  by  rating 
all  individuals  in  one  trait  before  rating  any  of  the  individuals 
in  any  other  trait. 

(7)  More  reliable  results  are  obtained  by  having  a  person 
rated  by  more  than  one  person,  ratings  by  three  persons  being- 
recommended  if  it  is  possible  to  get  them.  These  ratings  should 
be  made  independently  and  then  averaged.  A  revision  of  one 
rating  by  another  person  is  not  so  accurate  as  a  combination  of 
two  ratings  made  separately. 

2.  Evaluation  of  the  Rating  Scale  Used,  Entitled, 
"Rating  by  Interviewer" 

With  the  above  principles  in  mind,  a  study  was  made  of  the 
results  of  the  scale  already  in  use  by  the  Company,  which  is  en- 
titled Rating  by  Interviewer  (p.  24).  The  purpose  was  to 
check  the  value  of  such  a  scale  and  the  particular  selection  of 
traits.  The  scale  was  used  with  only  thirty-one  students,  but  the 
number  is  sufficient  to  give  results  that  are  significant,  though  by 
no  means  final.     Table  VI.  shows  the  correlations  between  the 
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RATING  BY  INTERVIEWER 
THIS  SIDE  NOT  TO  BE  WRITTEN  UPON  BY  APPLICANT 


I.  Physical  qualities — Physique, 
bearing,  neatness,  voice,  energy, 
endurance.  (Consider  how  he  im- 
presses men  in  the  above  respects.) 


II.  Intelligence — Accuracy,  ease 
in  learning,  ability  to  grasp  the 
point  quickly,  to  express  himself 
clearly,  and  to  estimate  a  new  sit- 
uation. 


III.  Leadership  —  Initiative, 
force,  decisiveness,  tact,  helpful- 
ness, ability  to  inspire  men  and  to 
win  their  loyalty  and  cooperation. 


IV.  Personal  qualities — Indus- 
try, dependability,  loyalty,  readiness 
to  shoulder  responsibility  for  his 
own  acts,  freedom  from  conceit  and 
selfishness,  readiness  and  ability  to 
cooperate,  and  charm  of  personal- 
ity, breadth   of  conceptions. 


V.  General  Value  to  the  Com- 
pany— Special  talent  (commercial, 
mechanical,  mathematical,  execu- 
tive), gets  results,  works  well  in 
an  organization.  .Special  interest 
in  W.  E.  &  M.  Co.  Previous  train- 
ing and   experience.     Will   develop. 


Highest 

20 

High 

16 

Average 

12 

Low 

8 

Lowest 

4 

Highest 

20 

High 

16 

Average 

12 

Low 

8 

Lowest 

4 

Highest 

20 

High 

16 

Average 

12 

Low 

8 

Lowest 

4 

Highest 

20 

High 

16 

Average 

12 

Low 

8 

Lowest 

4 

Highest 

20 

High 

16 

Average 

12 

Low 

8 

Lowest 


Remarks 


1 


Recommendations  of  Professors  and   Instructors — 


REPORT  OF  INTERVIEWER 


1.     General   impressions. 


2.  Work  best  suited   for 

3.  Acceptance    (write  Yes   or  No) 

4.  Next   action   promised   by    interviewer. 


Signed 


Interviewer 


Date. 


« 
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original  interview  ratings,  the  tests,  and  the  ratings  made  by 
the  Educational  Department  committee  one  year  later.  Also 
the  relation  between  each  trait  and  the  total  of  ratings  on  each 
trait,  and  the  intercorrelations  between  traits  is  shown.  All  cor- 
relations are  positive. 

Table  VI. 
Correlations  of  Ratings  by  Interviewers  with  Other  Criteria. 

Number     Correlation 
of  cases  (r) 

Interviewers'    composite    ratings    with    estimated    in- 
telligence, committee's  rating   31  +.65 

Interviewers'    composite    ratings    with    Bureau    Test 

VI   scores    18  +-58 

Interviewers'  ratings  of  intelligence  with  Bureau  Test 

VI   scores    18  +-57 

Interviewers'    ratings    of   intelligence   with   estimated 

intelligence,   committee's   rating 31  -\-.22 

Interviewers'  composite  ratings  with  shop  ratings 27  .0 

Interviewers'  ratings  of  intelligence  with  shop  ratings 

on  analytical  ability   26  .0 

Interviewers'  ratings  of  personal  qualities  with  shop 

ratings   on  personality    27  -f  .25 

Interviewers'  composite  ratings  with  technical  school 

grades   21  +.22 

Interviewers'  composite  ratings  with  their  ratings  of 

leadership     32  +.80 

Interviewers'  composite  ratings  with  their  ratings  of 

personal  qualities    32  +79 

Interviewers'  composite  ratings  with  their  ratings  of 

physical   qualities    32  +73 

Interviewers'  composite  ratings  with  their  ratings  of 

general  value  to  company  32  +.68 

Interviewers'  composite  ratings  with  their  ratings  of 

intelligence    32  +.68 

Personal  qualities  with  general  value  to  company 32  -\--^7 

Personal  qualities  with  leadership    32  +-59 

Intelligence  with  general  value  to  the  company 32  +48 

Leadership  with  general  value  to  the  company  32  +47 

Leadership  with  physical  qualities    32  +46 

Personal  qualities  with  physical  qualities  32  +45 

Intelligence  with  personal  qualities  32  +-44 

Physical  qualities  with  general  value  to  company 32  +43 

Intelligence  with   leadership    32  +40 

Intelligence  with  physical  qualities   32  -\--2$ 

The  interviewers'  ratings  of  intelligence  agree  much  better 
with  the  scores  of  the  psychological  test  for  intelligence,  r  = 
+.57,  than  they  do  with  the  committee's  later  estimate  of  intel- 
ligence, r  =  +.25;  but  since  the  interviewers'  ratings  of  intelli- 
gence correlate  comparatively  low  with  their  composite  ratings 
including  other  qualities,  r  =  +.68,  and  since  the  composite 
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ratings  have  high  correlation  with  the  later  estimate  of  intelli- 
gence by  the  committee,  r  =  +.65,  we  might  take  these  facts 
as  evidence  that  qualities  other  than  intelligence  have  uncon- 
sciously been  included  in  the  later  ranking  of  the  men. 

The  most  important  correlations  to  consider  in  relation  to  the 
rating  scale,  are  those  of  Personal  Qualities,  which  correlate, 
with  General  Value  to  the  Company,  r  =  +.67,  and  with  Lead- 
ership, r  =  +-59-  These  correlations  are  entirely  too  high  be- 
tween traits  which  according  to  the  best  principles  should  be 
disparate  traits.  These  high  correlations  are  undoubtedly  due  to 
the  indefinite  and  too  inclusive  definitions  of  these  traits.  There- 
fore I  have  worked  to  make  these  traits  more  definite  and  ex- 
clusive in  the  revised  definitions  or  descriptions  of  them.  The 
revised  forms  are  seen  in  the  Interviewer's  Master  Scale,  dis- 
cussed later.  Some  of  the  other  traits  have  had  the  definitions 
slightly  changed  to  make  the  same  improvement  toward  definite 
unit  traits  which  could  be  more  accurately  rated. 

3.     The  Revised  Form  of  Interviewer's  Rating  Scale: 

The  Interviewer's  Master  Scale  ' 

Not  only  were  these  results  considered  in  the  revision,  but  the 
whole  aim  was  to  make  the  rating  scale  best  adapted  to  select 
men  for  the  particular  lines  of  work  to  be  done  for  the  particular 
Company.  From  the  occupational  description  or  information  that 
had  been  obtained  concerning  each  line  of  work  to  which  a  grad- 
uate engineer  is  later  assigned,  I  made  an  analysis  of  what  quali- 
fications or  traits  are  essential  for  success  in  each  line  of  work 
or  specific  occupation.  The  traits  for  each  occupation  were 
listed  separately.  Then  to  enrich  this  list  and  to  be  sure  that  no 
traits  were  omitted,  comparison  was  made  with  the  list  of  traits 
already  being  used  by  the  Westinghouse  interviewers  and  also 
with  the  traits  considered  in  a  rating  made  later  in  the  Educa- 
tional Department.  Also  all  other  available  rating  scales,  includ- 
ing the  Bureau  Scale  for  Salesmen,  and  the  Army  Officer  Rating 
Scale,  were  compared  with  the  list  of  traits.  Finally,  Mann's 
Study  of  Engineering  Education,  and  Davenport's  Trait  Book, 
which  lists  several  hundred  traits,  were  examined  for  suggestions 
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of  traits.  From  the  final  nine  lists  of  traits  (one  list  for  each 
occupation),  selection  was  made  for  the  traits  common  to  all 
lists  and  those  which  were  similar  were  grouped  under  one  name 
of  an  essential  trait.  Then  selection  was  made  of  those  traits 
which  were  not  common  and  which  tended  to  differentiate  the 
occupations.  For  making  the  scales  to  be  used  in  the  first  selec- 
tion and  employment  of  applicants  or  prospective  applicants,  the 
common  and  the  differentiating  traits  were  combined.  For  con- 
structing the  scales  which  were  to  be  used  later  in  separating  the 
men,  the  differentiating  traits  were  emphasized. 

The  first  work  in  preparing  rating  scales  was  to  revise  the  scale 
already  in  use,  called  the  Interviewer's  Rating,  shown  on  page  24. 
As  the  revisions  were  simply  in  the  definitions  of  the  traits,  these 
may  be  seen  in  the  form  for  the  Interviezver's  Master  Rating 
Scale,  which  was  prepared.  This  Master  Scale  is  the  standard 
or  measuring  rod  with  which  the  senior  engineer  is  compared 
and  rated  by  the  interviewer.  The  form  of  this  is  shown  on  page 
28.  The  method  of  making  and  using  the  scale  is  explained  in 
the  following  section. 

4.  Instructions  to  Interviewers  for  Rating 
Senior  Engineers 

Significance  of  the  Rating  Scale. 

(1)  The  rating  of  a  senior  engineer  is  a  numerical  expression 
of  the  degree  in  which  he  possesses  five  qualifications  consid- 
ered essential  in  a  Westinghouse  engineer,  namely:  (I.)  Physical 
Qualities,  (II.)  Intelligence,  (III.)  Leadership,  (IV.)  Social  and 
Personal  Qualities,  (V.)  General  Value  to  the  Company.  The 
rating  is  made  by  comparing  him  in  each  of  these  respects  with 
engineers  who  are  now  employed  in  the  Company.  Each  rater 
makes  his  own  scale,  using  the  Interviewer's  Master  Rating  Scale. 
Proper  rating  is  largely  dependent  on  the  possession  of  an  ac- 
curate Master  Rating  Scale.  Do  not  start  to  prepare  your  scale 
until  you  can  give  at  least  thirty  minutes  to  it. 
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INTERVIEWER'S  MASTER  SCALE 


\ 


(To  be  filled  out  by  all  interviewers  in  committee  together,  for  use  as  a 
standard   in   rating   each   applicant.) 

Have  before  you  the  names  of  at  least  twenty-five  of  your  engineers  and 
salesmen  with  whom  you  all  are  well  acquainted.  Include  in  this  list 
men  that  rank  highest,  lowest  and  intermediate  in  Physical  Qualities;  men 
that  rank  highest,  lowest  and  intermediate  in  Intelligence ;  and  similarly  in 
Leadership,  in  Social  and  Personal  Qualities,  and  in  General  Value  to  the 
Company. 


I.  Physical  Qualities 
Disregard  every  characteristic  of 
each  of  your  engineers  and  sales- 
men except  their  physical  qualities. 
Consider  physique,  bearing,  neat- 
ness, facial  expression,  voice,  charm 
of  appearance  and  manner,  energy, 
and  endurance.  (Consider  how  he 
impresses  men  in  such  qualities.) 

Highest  Mr 20 

High        Mr 16 

Middle     Mr 12 

Low         Mr 8 

Lowest    Mr 4 

II.  Intelligence 

Consider  common  sense,  judgment, 
ability  to  grasp  the  point  quickly,  to 
express  himself  clearly,  and  to  esti- 
mate a  new  situation.  On  the  basis 
of  intelligence  make  a  selection  of 
five  of  your  men  as  in  Number  I 
above, — highest,  lowest,  middle, 
high,  low.  These  five  need  not  be 
the  same  as  those  in  Number  I. 

Highest  Mr 20 

High        Mr 16 

Middle    Mr 12 

Low         Mr 8 

Lowest  Mr 4 


III.  Leadership 

Consider  force,  decisiveness,  enthus- 
iasm, and  ability  to  inspire  men  and 
to  win  their  loyalty  and  cooperation. 

Highest  Mr 20 

High        Mr 16 

Middle     Mr 12 

Low         Mr 8 

Lowest    Mr 4 

IV.  Social  and  Personal  Qualities 
Consider  conversational  ability,  tact, 
freedom  from  conceit  and  selfish- 
ness, readiness  to  shoulder  responsi- 
bility for  own  acts,  readiness  and 
ability  to  cooperate,  sociableness, 
and  congeniality. 

Highest   Mr 

High        Mr 

Middle    Mr 

Low         Mr 

Lowest    Mr 


20 

16 

12 

8 

4 


V.  General  Value  to  the  Company 
Consider  previous  training  and  ex- 
perience showing  interest  and  adap- 
tability, (commercial,  mechanical, 
mathematical,  executive)  special  in- 
terest in  W.  E.  &  M.  Co.  and  pro- 
fessional attitude. 

Highest  Mr 20 

High        Mr 16 

Middle     Mr 12 

Low         Mr 8 

Lowest    Mr 4 


\ 


How  to  Make  the  Master  Scale. 

(2)  Write  on  small  slips  of  paper  the  names  of  from  12  to  25 
engineers  who  have  passed  through  the  Educational  Department 
within  the  previous  five  years.  They  should  be  men  with  whom 
you  are  well  acquainted,  and  of  whose  degree  of  success  you  have 
some  definite  knowledge.  Include  some  whose  qualifications  are 
extremely  poor  as  well  as  those  who  are  highly  efficient.    If  these 
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names  do  not  include  all  the  grades  for  each  of  the  five  qualificaj 
tions,  others  may  be  added. 

(3)  Look  over  your  names  from  the  viewpoint  of  Physical 
Qualities  only.  Disregard  every  other  characteristic  of  each 
man  except  the  way  in  which  he  impresses  men  by  his  physique, 
bearing,  neatness,  voice,  charm  of  appearance  and  manner,  en- 
ergy and  endurance.  Arrange  the  names  on  the  slips  of  paper  in 
order  from  highest  to  lowest  on  the  basis  of  the  Physical  Quali- 
ties of  the  men.  Select  that  engineer  who  surpasses  all  the  others 
in  this  qualification  and  enter  his  name  on  the  line  marked 
Highest  under  Physical  Qualities  on  the  Interviewer's  Master 
Scale.  Then  select  the  one  who  most  conspicuously  lacks  these 
qualities  and  enter  his  name  on  the  line  marked  Lowest.  Select 
the  man  who  seems  about  half  way  between  the  two  previously 
selected  and  who  represents  about  the  general  average  in  Phys- 
ical Qualities;  and  enter  his  name  on  the  line  marked  Middle. 
Select  the  engineer  who  is  halfway  between  middle  and  highest, 
and  enter  his  name  on  the  line  marked  High.  Select  the  one  who 
ranks  halfway  between  middle  and  lowest,  and  enter  his  name  on 
the  line  marked  Low. 

(4)  In  the  same  manner  make  out  scales  for  each  of  the  other 
four  qualifications  (Intelligence,  Leadership,  Social  and  Personal 
Qualities,  and  General  Value  to  the  Company). 

(5)  Each  engineer  whose  name  appears  on  the  Scale  should 
be  one  who  exhibits  clearly  and  distinctly  the  qualification  and 
the  degree  of  the  qualification  for  which  he  has  been  chosen. 

(6)  The  names  for  Highest  and  Lowest  on  each  section  of 
the  Scale  must  represent  extreme  cases.  The  names  for  the 
Middle  should  be  that  of  an  average  engineer,  halfway  between 
extremes.  High  and  Low  should  be  halfway  between  the  Middle 
and  the  extremes.     An  even  gradation  of  merit  is  important. 

(7)  In  making  or  using  any  section  of  the  Scale,  consider 
only  the  qualification  it  covers,  totally  disregarding  all  the  others. 

How  to  Use  the  Scale. 

(8)  Rate  your  senior  student  engineer  for  Physical  Qualities 
first.     Consider  how  he  impresses  men  by  his  physique,  bearing, 
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\ 
neatness,  voice,  charm  of  appearance  and  manner,  energy,  and 

endurance.  Compare  him  with  each  of  the  five  men  in  Section  I. 
of  your  Master  Rating  Scale,  and  give  him  the  number  of  points 
following  the  name  of  the  man  he  most  nearly  equals.  Check  this 
number  under  Physical  Qualities  on  your  blank  entitled  Inter- 
viewer's Rating.  If  he  falls  between  two  men  in  the  Scale,  give 
him  a  number,  accordingly,  between  the  numbers  of  these  two 
names  in  the  Scale. 

(10)  Rate  the  engineers  in  a  corresponding  manner  for  each 
of  the  other  four  essential  qualifications. 

(n)  In  rating,  make  a  man-to-man  comparison  of  the  man 
with  the  men  whose  names  appear  on  your  Scale — never  in 
terms  of  numbers  directly.  Disregard  the  numerical  equivalent 
until  you  have  made  these  concrete  comparisons. 

(12)  When  rating  several  engineers,  rate  all  of  them  on  each 
qualification  before  adding  the  total  for  any  one. 

(13)  This  is  not  a  percentage  system  and  you  should  not 
allow  yourself  to  fix  in  mind  any  particular  number  of  points 
you  think  the  engineer  ought  to  get. 

(14)  The  total  rating  for  an  engineer  is  the  sum  of  the  ratings 
you  give  him  in  the  five  separate  qualifications. 

5.     College  Instructor's  Rating  Scale. 

There  are  many  traits  desirable  in  an  engineer  which  can  not 
be  rated  with  any  reliability  from  a  brief  interview  with  a  senior 
student.  However,  an  instructor  having  had  the  student  in  his 
classes  and  knowing  of  his  college  activities,  could  rate  the  man 
on  definite  objective  criteria.  It  was  thought  at  first  that  the 
College  Instructor's  Rating  Scale  (see  page  31)  should  have 
entirely  different  traits;  but  in  order  to  reduce  the  possibilities 
of  confusion  to  the  minimum  and  to  make  the  combination  of  the 
ratings  of  the  two  scales  easy,  practically  the  same  names  of 
traits  are  used  on  both  scales.  The  ratings  from  the  Instructor's 
Rating  Scale  can  be  checked  just  to  the  left  of  the  interviewer's 
ratings  on  the  present  form,  Rating  by  Interviewer.  However, 
it  is  important  that  the  interviewer  make  his  rating  independently 
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COLLEGE  INSTRUCTOR'S  RATING  SCALE 

Will  you  please  rate  the  student  named  below  for  the  traits  indicated, 
keeping  in  mind  employment  after  graduation.  Give  the  rating  independently 
without  consulting  others.  This  rating  sheet  is  to  be  returned  to  the  Bureau 
of  Recommendations  without  delay.  Do  not  rate  any  student  whom  you 
have  not  had  in  class  at  least  one  term. 

Name  

Among  the  members  of  the  average  senior  class  in  this  student's  course 
and  school,  in  which  fifth  would  this  student  rank  in  each  trait?  Indicate 
the  rank  in  each  trait  by  placing  a  check  (  V  )  at  the  proper  height  in  each 
column,  grading  the  student  as  finely  as  you  can.  The  traits  are  described 
as  follows : 

I.  Appearance  and  Manner. — Consider  physique,  bearing,  neatness,  facial 
expression  and  voice.     (Consider  how  he  impresses  men  in  such  qualities.) 

II.  Intelligence. — Consider  accuracy,  promptness,  ease  of  learning,  ability 
to  grasp  the  point  quickly  and  to  express  himself  clearly,  and  ability  to  esti- 
mate a  new  situation  and  to  get  desired  results  professionally. 

III.  Leadership. — Consider  initiative,  self-reliance,  enthusiasm,  tact,  and 
ability  to  inspire  men  and  to  win  their  loyalty  and  cooperation. 

IV.  Personal  Qualities. — Consider  honesty,  reliability,  spirit  of  service, 
sense  of  responsibility  for  his  own  acts  and  moral  stamina. 

V.  Professional  Interest. — Consider  industry,  energy,  perseverance,  and 
conscientious  application  to  work. 


Appearance 
and  Manner 

Intelligence 

Leadership 

Personal 
Qualities 

Profession- 
al Interest 

Highest   Fifth 

Second  Fifth 

Middle    Fifth 

Fourth  Fifth 

Lowest  Fifth 

1 

To  aid  further  the  Bureau  of  Recommendations,  please  indicate,  if  possible, 
the  specific  kind  of  work  for  which  this  student  is  best  fitted. 


In  this  Specific  Work  how  would  you  rate 
this  student  among  the  members  of  the  aver- 
age Senior  Class  in  this  student's  Course 
and  School?  Indicate  your  opinion  by  a 
check  ( V )  at  the  proper  height  in  the  col- 
umn to  the  right. 


Specific   Work 

Highest  Fifth: 

Second  Fifth: 

Middle  Fifth : 


Fourth  Fifth 


Lowest  Fifth  : 


(Use  the  other  side  of  this  sheet  for  remarks  concerning  the  student.) 
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and  before  he  sees  the  instructor's  ratings  or  copies  them  on  liis 
sheet.    Then  an  average  of  the  two  can  be  made. 

The  traits  to  be  rated  are  not  defined  the  same  on  the  two 
scales.  They  should  be  rated  from  two  different  viewpoints, 
namely  from  what  the  instructor's  acquaintance  and  observation 
reveals  to  be  the  actual  rating  of  the  trait,  and  from  the  first  im- 
pression which  the  individual  gives  other  people  regarding  the 
trait,  as  shown  by  the  interviewer's  rating.  With  the  background 
and  viewpoint  of  the  instructor  differing  from  that  of  the  inter- 
viewer, the  ratings  will  supplement  and  balance  or  check  each 
other.  Professional  interest  or  industry  and  professional  ability 
can  be  judged  better  by  the  college  instructor;  but  General  Value 
to  the  Company  can  be  judged  better  by  the  Company  representa- 
tive. 

The  instructor  cannot  be  expected  to  prepare  a  scale,  so  he  is 
asked  to  rate  the  student  in  terms  of  fifths  of  the  average  senior 
class.  The  interviewers,  however,  should  prepare  an  Inter- 
viewer's Master  Rating  Scale,  and  by  mental  comparison  of 
students  with  the  men  named  on  this  scale,  that  is,  a  man-to-man 
comparison,  a  rating  can  be  given  to  that  student  for  each  trait, 
and  then  be  entered  on  the  form,  Rating  by  Interviewer. 

6.     Shop  and  Class  Rating  Scale 

The  unreliability  of  the  shop  ratings  made  by  the  foremen,  and 
of  the  class  and  shop  grades,  has  already  been  discussed  in 
Chapter  VII,  entitled  "Evaluation  of  Grades  and  Ratings  Given 
During  Training  in  the  Educational  Department  of  the  Indus- 
trial Firm."  It  was  stated  there  that  the  causes  of  this  unre- 
liability were  evidently  due  to  the  lack  of  adequate  definitions  of 
the  traits,  to  the  lack  of  common  understanding  by  the  foremen 
of  the  meaning  of  the  names  of  the  traits,  to  the  lack  of  a  stan- 
dard, scale,  or  measuring  rod  with  which  to  compare  the  persons 
being  rated,  and  to  the  arbitrary  method  of  recording  and  re- 
porting the  ratings.  In  order  to  obviate  or  at  least  minimize 
these  causes  of  unreliability  of  the  ratings,  the  Shop  and  Class 
Rating  Scale  was  prepared.  It  is  intended  that  each  foreman  and 
instructor  shall  rate  each  student  engineer  under  them  once  each 
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Name  Section 

Rate  the  man  named  above  in  the  traits  indicated.  The  traits  are  defined 
and  described  on  the  other  side  of  this  sheet. 

Among  the  Graduate  Engineers  whom  you  have  known  in  this  particular 
work  that  you  have  charge  of,  in  which  fifth, — highest,  second,  middle,  fourth, 
or  lowest,  should  this  man  rank  in  each  trait?  Indicate  the  rank  in  each  trait 
by  placing  a  check  (  V  )  at  the  proper  height  in  each  column,  grading  the  man 
as  carefully  as  you  can. 


Intelligence 

Co- 
operation 

Industry 

Leadership 

Highest  Fifth  

Second  Fifth    

Middle  Fifth   

Fourth  Fifth    

Lowest  Fifth   

In  what  work  does  this  man  excel?. 


In  what  is  he  deficient?. 


Signed  

Note  : — Send  to  the  Educational  Department. 


Date 


DEFINITIONS  OF  TRAITS  OR  QUALITIES  TO  BE  TREATED 

I.  Intelligence. — Consider  aptitude,  ability  to  learn,  common-sense,  judg- 
ment, and  ability  to  grasp  the  point,  to  express  himself  clearly,  and  to  deal 
with  a  new  situation. 

II.  Cooperation. — Consider  tact,  courtesy,  fair-mindedness,  sociability,  and 
ability  to  get  along  with  other  men  and  to  fit  into  the  organization. 

III.  Industry. — Consider  energy,  perseverance,  conscientious  application, 
reliability,  interest,  enthusiasm,  and  loyalty. 

IV.  Leadership. — Consider  initiative,  aggressiveness,  force,  decisiveness, 
and  ability  to  influence  men  and  win  their  loyalty  and  cooperation. 

month  on  this  Shop  and  Class  Rating  Scale.  The  ratings  are  to 
be  sent  to  the  Educational  Department  office,  where  they  are  to 
be  combined  with  the  other  data  on  the  student  engineers. 

For  differentiating  the  sales  engineers  from  the  other  en- 
gineers certain  traits  considered  essential  for  sales  engineering 
ability  can  be  selected  and  the  total  rating  on  these  traits  for  each 
graduate  engineer  will  show  his  probable  aptitude  for  sales 
engineering.  To  take  advantage  of  data  previously  obtained, 
ratings  on  certain  traits  which  differentiate  the  graduate  en- 
gineers, for  example  traits  desired  in  a  salesman,  might  be  se- 
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(1 
lected  from  the  other  rating  scales  and  combined  with  these.    For 

example,  ratings  on  leadership,  social  and  personal  qualities,  and 

physical  qualities  might  be  taken  from  the  Interviewer's  Rating 

Scale  and  from  the  College  Instructor's  Rating  Scale. 

VIII.     Survey  and  Evaluation  of  Interests  as 
Criteria  for  Vocational  Placement 

i  .     The  Significance  of  Interests 

The  technical  school  grades,  the  grades  and  ratings  on  the 
work  of  the  men  during  training  in  the  Educational  Department, 
and  the  results  of  the  tests  discussed  in  the  next  section,  indicate 
that  the  engineer  cannot  be  differentiated  for  the  different  kinds 
of  work  by  general  intelligence  or  mental  alertness.  Men  with 
equal  mental  ability  are  found  in  all  the  lines  of  work.  Suc- 
cess of  a  certain  person  in  a  particular  line  of  work  is  evidently 
due  to  general  intelligence  or  mental  ability  directed  in  a  par- 
ticular line  by  interests.  However,  it  is  probably  true  that  the 
interests  are  based  on  a  particular  ability  or  group  of  abilities 
which  makes  activity  and  achievement  in  a  particular  line  of  4 

work  possible  and  interesting.  Leaving  the  question  of  general 
intelligence  and  particular  abilities  to  be  discussed  in  the  next 
section,  we  consider  here  the  interests  themselves  without  rais- 
ing the  question  of  what  causes  them. 

After  the  man  has  a  definite  strong  interest  in  any  line  of  work 
or  activity,  this  interest,  motivation,  or  whatever  it  may  be,  should 
be  taken  advantage  of.  It  is  difficult  to  discover  just  what  the 
interests  are,  and  often  the  graduate  student  engineers  cannot 
decide  themselves  what  work  they  should  prefer.  In  order  to 
make  possible  the  planning  and  direction  of  their  training,  the 
custom  has  been  to  ask  them  to  indicate  their  choice  of  work  at 
the  end  of  two  months'  work  in  the  Westinghouse  Company, 
during  which  time  they  have  been  doing  various  kinds  of  work 
in  the  shop  under  the  supervision  of  the  Educational  Department. 
To  help  them  make  this  choice,  and  to  help  the  executives  know 
more  definitely  what  this  choice  is,  the  form,  Record  of  Inter- 
ests, which  is  a  kind  of  rating  scale,  has  been  prepared. 
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2.  Method  of  Using  the  Record  of  Interests 
With  the  exception  of  Section  II.,  entitled  Choice  of  Other 
Occupations,  the  whole  of  the  Record  of  Interests  blank  may 
appear  to  add  nothing  new  to  the  ordinary  questions  often  asked 
when  attempting  to  place  a  man.  Nevertheless,  there  is  a 
definite  attempt  to  get  these  facts  tabulated  in  usable  form.  Each 
class  of  information  or  facts  concerning  the  man  is  considered 
separately,  and  a  definite  unmixed  judgment  is  made  on  that 
phase  of  the  man's  interests  and  qualifications.  Thus  the  Record 
of  Interests  is  really  a  rating  scale  of  interests.     The  judgments 

RECORD  OF  INTERESTS 

Prepared  by  B.  V.  Moore,  Research  Fellow,  Expressly  for  the  Westinghouse 

Electric  &  Manufacturing  Company,  East  Pittsburgh,  Pennsylvania. 

To  the  Graduate  Student  Engineer: 

Answer  the  questions,  and  fill  in  the  blanks  as  accurately  as  you  can.  None 
of  your  statements  here  commit  you  to  any  particular  assignment.  This 
record  enables  us  to  help  you  find  the  most  suitable  position  in  the  Westing- 
house  organization. 

I.     PREVIOUS  VOCATIONAL  INTERESTS 

1.  In  your  college  work  which  type  of  work  or  subjects  interested  you 
most?  Rank  the  types  of  work  below  in  order  of  pieference,  numbering 
the  best  liked  (1),  the  next  best  (2),  and  so  on  to  the  least  liked,  which 
you  number  (9). 

(     )  English  (     )  Field  Work  (     )  Testing 

(     )   Economics  (     )   Shop  Work  (     )   Laboratory 

(     )   History  (     )  Drawing  or  Draft'g        (     )   Mathematics 

2.  Did  you  ever  construct  or  build  any  mechanical  or  electrical  toy,  appara- 
tus, or  machines  of  any  kind  before  you  entered  college? 

(     )   No  (     )  Yes 

3.  What  were  the  things  you  made  ? 

4.  Did  you  make  these  things  chiefly  because  you  wanted  the  things  them- 
selves or  because  you  enjoyed  making  them? 

(     )  Wanted  the  things.  (     )  Enjoyed  making  them. 

5.  In  making  the  toy,  apparatus,  or  machine,  did  you  make  it  like  another, 
or  did  you  design  it  yourself? 

(     )   Made  it  like  another.  (     )   Designed  it  myself. 

6.  Did  you  ever  work  as  a  salesman  or  clerk  in  a  store,  sell  in  a  house 
to  house  canvass,  or  work  as  a  salesman  in  any  way? 

(     )  Yes  (     )   No 

7.  What  do  you  consider  to  be  the  most  responsible  position  that  you  have 
ever  held? 

8.  In  all  your  practical  experience,  including  remunerative  work  during  va- 
cations, which  work  or  particular  job  have  you  liked  best? 

9.  Why  did  you  choose  engineering  as  your  profession? 
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II.     CHOICE  OF  OTHER  OCCUPATIONS 

Disregarding  all  your  training  and  education  and  all  differences  in  com- 
pensation and  social  standing  of  the  following  occupations,  consider  only 
your  interest  and  satisfaction  in  working  at  each  of  the  following  occu- 
pations represented.  Check  with  a  plus  (+)  the  ten  kinds  of  work  which 
you  would  most  prefer  to  do;  and  check  with  a  minus  ( — )  the  ten  which 
you  would  dislike  or  least  like  to  do. 


Architect 

Automobile  repairman 

Automobile  salesman 

Bank  cashier 

Carpenter 

Draftsman 

Editor  of  popular  magazine 

Hotel  keeper  or  owner 

U.  S.  Governm't  astronomer 

Lawyer 


Machinist 

Newspaper  reporter 

Pattern-maker 

Private  secretary 

Purchasing  agent 

Real  estate  agent 

Research  worker  in  physics 

Stockbroker 

Toolmaker 

Watchmaker 


III.     AVOCATIONAL  INTERESTS 

i.    What  do  you  like  to  do  as  an  avocation,  hobby,  or  sideline? 

2.  What  form  of  recreation  or  entertainment  do  you  enjoy  most  in  the 
evening  after  a  day  of  study  or  work? 

3.  What  sports  do  you  enjoy  most  as  a  participator? 

4.  What  professional  and  business  magazines  do  you  read  regularly? 


)  Yes 

( 

)  No 

)  Yes 

f 

)  No 

)  Yes 

( 

)  No 

)  Yes 

( 

)  No 

5.    Have  you  participated  in  any  of  the  following  extra-academic  activities 
Student  debating  (     )  Yes  (     )   No 

Dramatics  ( 

Student  paper  or  annual  book  ( 

School  politics  ( 

Ever  captain  of  a  team  ( 

IV.  SOCIAL  INTERESTS 

1.  Name   the   social   clubs,    fraternities,   and    organizations   to    which   you 
belong. 

2.  Have  you  ever  held  any  important  office  in  any  of  these  organizations? 

(     )  Yes  (     )  No 

3.  How  many  times  have  you  made  an  after-dinner  talk? 

4.  While  you  were  in  college  did  you  prefer  to  live  with  a  roommate  or 
did  you  prefer  to  room  alone? 

(     )   Roommate  (     )  Room  alone 

5.  Do  you  usually  have  a  good  time  at  smokers,  conventions,   and  other 
gatherings  of  men? 

(     )  Yes  (     )   No 

6.  Excluding    engineering    magazines,    name    three    magazines    which    you 
enjoy  reading. 

V.  INTERESTS  OF  RELATIVES 

1 .  What  is  or  was  the  occupation  of  your  father  ? 

2.  What  were  the  occupations  of  your : 

(a)  Father's    father? 

(b)  Mother's  father  ? 

3.  What  are  occupations  of  your  brothers? 
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VI.    TECHNICAL  INTERESTS  IN  WESTINGHOUSE  ORGANIZA- 
TION AND  ITS  WORK 

1.  Considering  all  aspects  of  each  line  of  work  as  you  have  become  ac- 
quainted with  them  here  so  far,  decide  which  particular  kind  of  work  you 
would  prefer  to  make  your  life  occupation.  Give  this  line  of  work  a  rank 
or  score  of  (i).  Give  your  second  choice  a  rank  of  (2),  and  so  on, 
giving  your  least  desirable  choice  a  rank  of  (4). 

(     )   Sales  (     )   General  Engineering 

(     )  Works  Management  (     )   Design  Engineering 

2.  Considering  the  content  or  subject-matter  of  the  work  itself,  which  sub- 
ject or  courses  in  the  W.  E.  &  M.  Co.  Educational  Department  have  in- 
terested you  most?    Write  them  in  order  of  preference,  the  best  liked  first. 

(1) 

3.  Check  with  a  plus  (+)  the  following  lines  of  work  which  you  think  you 
would  choose  as  your  work  in  the  Westinghouse  organization.  Check 
with  a  minus  ( — )  those  which  you  think  you  would  not  choose  as  your 
work  in  the  Westinghouse  organization. 


(  ) 

General  Engineering 

Railway 

Sales 

(     )  Transformer  Engineering            (     )  Marine  Sales 

(     )   Control  Engineering                      (     )   Stocker   Sales 

(     )   Switchboard  Engineering             (     )   Foreign    Sales 

(     )   Industrial  and  Railway                (     )   Production 

Motor  Engineering                   (     )  Time  and  Motion  Study 

(     )   Railway  Equipment                       (     )  Inspection 

Engineering                                (     )   Cost  Accounting 

(     )   Material  and  Process                    (     )   Maintenance 

Engineering                                (     )  Operating 

(     )   Research  Engineering                   (     )   Dynamo  Test 

(     )   Detail  Engineering                        (     )   Transformer  Test 

(     )   Power  Engineering                       (     )  Large  Industrial  Motor  Test 

(     )   Industrial  Sales                              (     )   Small  Industrial  Motor  Test 

(     )   Supply  Sales                                   (     )  Detail  Test 

(     )   Power  Sales                                    (     )  Farm  Lighting  Equipment 

Test 

(Do  not  write  in  the  space  below) 

I. 

II.         |         III               IV. 

V. 

Vocational 

Other  Occ'sl  Avocations 

Social 

Relatives 

Technical 

Design 

General 

Oper'g 

Service 

Works 

Sales 

I 

REMARKS : 
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or  ratings  are  made  by  a  man  or  by  men  who  are  intimately 
familiar  with  the  lines  of  work  and  opportunities  open  to  the 
student  engineers.  With  these  lines  of  work  in  mind,  the  one 
doing  the  rating  considers  the  man's  answers  to  the  questions 
of  Section  I.,  entitled  Previous  Vocational  Interests,  and  decides 
whether  the  facts  appearing  there  indicate  that  the  man  is  quali- 
fied by  his  fundamental  interests  and  experience  to  be  a  design 
engineer,  a  general  engineer,  a  works  management  engineer,  an 
operating  engineer,  a  service  engineer,  or  a  sales  engineer.  Then 
turning  to  the  Summary  on  the  last  page  of  the  Record  of  In- 
terests, the  rater  places  a  check  (  \/ )  in  the  proper  rectangular 
blank  space  under  the  column  head,  I.  Vocational.  Each  section 
is  considered  separately  in  this  way,  a  judgment  is  made  from 
each  class  of  facts,  and  a  check  is  made  in  the  proper  square  of 
each  column  of  the  Summary.  By  connecting  the  check  marks 
with  straight  lines,  a  curve  is  obtained  which  shows  the  general 
tendency  of  the  interests  of  the  man,  whether  fitting  him  for 
design,  general,  works  management,  operating,  service,  or  sales 
engineering.  This  gives  a  definite  and  permanent  graphic  record 
of  pertinent  facts  concerning  each  man.  Instead  of  a  hazy  vacil- 
ating  general  impression  of  the  man,  there  is  a  tangible  analyzed 
judgment  of  him.  By  keeping  this  on  file  and  easily  available,  it 
is  possible  to  have  always  before  the  executive  considering  the 
man,  a  careful  record  of  previous  judgments. 

3.  Evaluation  of  Specific  Questions  in 
the  Record  of  Interests 
To  test  out  the  value  of  the  Record  of  Interests,  it  was  filled 
out  by  sixty-four  adults  in  one  class  and  sixty  in  a  second  class 
of  the  School  of  Insurance  Salesmanship  at  the  Carnegie  Insti- 
tute of  Technology.  The  results  for  each  Insurance  Salesman- 
ship class  are  tabulated  separately  in  order  that  the  two  sets  of 
data  will  be  a  check  on  each  other,  and  their  agreement  be  some 
measure  of  their  reliability.  Later  the  Record  of  Interests  was 
filled  out  also  by  thirty  Westinghouse  design  engineers  and 
thirty  sales  engineers,  none  of  whom  had  been  with  the  Com- 
pany less  than  a  year  nor  more  than  five  years.  The  following 
table  shows  the  results  obtained  for  some  of  the  chief  questions. 


PERSONNEL  SELECTION  OF  GRADUATE  ENGINEERS 


39 


Each  section  is  discussed  later  and  evaluated  separately  in  the 
light  of  the  results  obtained. 

Table  VII. 

Results  Obtained  from  Record  of  Interests  of  Insurance   Salesmen,   Design 

Engineers,  and  Sales  Engineers 

Insurance   Salesmen 

Answered  (Yes)  Answered  (No)     No.  of  Cases 
No.  of  First     Second     First      Second    First  Second 

Question       Question  Class      Class      Class      Class      Class    Class 

1.2.     Constructed  toy? 25%  40%  75%  60%  55  62 

I.4.     Wanted  toy?    27%  53%  73%  47%  IS  IS 

I.  5-     Copied   Design?    69%  47%  31%  53%  13  15 

1.6.     Had  sold  articles?....  84%  81%  16%  19%  62  52 

III.  5.     Student  Debating?    ...  51%  56%  49%  44%  51  45 

Dramatics?   45%  48%  55%  52%  53  46 

Student  paper  or  book?  40%  52%  60%  48%  53  46 

I.     1.  Rank  of  Choice  of  School  Studies  or  Subjects. 

Average  Rank  Given  Number  of  Cases 

First  Second  First  Second 

Class  Class  Class  Class 

English 3.07  3.23  56           39 

Economics     3-54  2.92  46            38 

History    300  3.29  53            45 

Field  Work   4.13  4.39  24            28 

Shop  Work   5.65  6.44  26            25 

Drawing  and   Drafting    5.19  7.13  32            24 

Testing    5-35  6.00  20           25 

Laboratory   4-97  5-*3  32            30 

Mathematics   347  3-6o  53            39 

Design   Engineers   and    Sales   Engineers 

Number 
No.  of  Answered  (Yes)  Answered  (No)     of  Cases 

Question        Question  Design     Sales    Design     Sales  Design  Sales 

I.  2.  Constructed  toy?    83%  50%  17%  5°%  30  46 

1.4.  For  toy  itself?    0%  37%  100%  63%  19  19 

1.5.  Copied  design?   12%  38%  88%  62%  17  16 

1.6.  Had  sold  articles?   50%  82%  50%  18%  25  45 

III.  5.  Student  debating?  32%  47%  68%  53%  25  45 

Dramatics?   16%        27%        84%        73%        25        45 

Student  paper  or  book?.  36%        33%        64%        67%        25        45 

I.     1.  Rank  of  Choice  of  School  Studies  or  Subjects. 

Average  Rank  Given  Number  of  Cases 

Design        Sales  Design     Sales 

English    6.73  5.33  26  39 

Economics     6.04  4.63  26  38 

History    6.32  5-55  25  36 

Field  Work    5-54  5*3  26  38 

Shop  Work    5-6"5  5-98  26  40 

Drawing  and  Drafting  5.31  5-95  26  39 

Testing    3-73  3-9S  26  40 

Laboratory    3-15  3-88  26  40 

Mathematics     1.81  385  27  40 
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I.  Previous  Vocational  Interests. — Question  I,  regarding  col- 
lege work,  proved  to  be  of  significance  in  differentiating  the  men. 
Both  the  insurance  salesmen  and  the  sales  engineers,  particularly 
the  insurance  salesmen,  liked  English,  Economics,  History,  and 
Field  Work  better  than  the  design  engineers  did.  The  design 
engineers  liked  Mathematics,  Laboratory,  Testing,  Drafting,  and 
Shop  better  than  the  sales  engineers  or  the  insurance  salesmen 
did. 

Questions  2  and  3  showed  some  tendency  for  this  differentia- 
tion, but  they  were  not  so  valuable.  Of  the  insurance  salesmen, 
25%  of  the  first  class  and  40%  of  the  second  class  had  con- 
structed toys;  50%  of  the  sales  engineers  had  constructed  toys, 
while  83%  of  the  design  engineers  had  constructed  toys.  Ques- 
tion 4  showed  a  tendency  of  the  salesmen  to  make  things  because 
they  wanted  them,  while  the  design  engineers  made  things  be- 
cause they  enjoyed  making  them.  Those  who  made  things  be- 
cause they  enjoyed  making  them,  were  73%  of  the  first  class 
of  insurance  salesmen  and  47%  of  the  second  class  of  insurance 
salesmen  who  made  any  things.  30%  of  the  sales  engineers  who 
made  things,  and  100%  of  the  design  engineers  who  made  things  * 

made  them  for  the  pleasure  of  construction.  The  other  questions 
of  this  section  were  not  so  reliable,  but  they  served  to  aid  in  form- 
ing a  judgment  or  classification  of  the  men. 

II.  Choice  of  Other  Occupations. — In  this  list  of  occupations, 
there  were  fifteen  occupations  which  were  judged  by  fourteen 
people  to  be  occupations  that  would  be  chosen  or  preferred  by  a 
man  with  engineering  interests  as  primary.  The  other  fifteen 
occupations  were  judged  by  the  same  people  to  be  occupations 
that  would  be  chosen  by  a  salesman  type  of  man.  The  occupa- 
tions were  carefully  chosen  from  a  much  larger  list;  and  the 
judgments  or  classifications  of  the  occupations  by  each  of  the 
fourteen  people  were  tabulated  so  that  any  occupation  concerning- 
which  there  was  not  common  agreement  was  detected  and  elim- 
inated from  the  list.  It  might  be  said  that  the  occupations  in- 
tended to  be  chosen  by  a  sales  type  of  person  are  comparatively 
higher-level  occupations  than  those  intended  to  be  chosen  by  the 
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engineer  type.  This  condition  was  allowed  to  exist,  because  all 
the  men  choosing  these  occupations  are  engineers  and  are  already 
inclined  to  choose  engineering  type  of  occupations.  By  having 
them  choose  from  a  list  containing  very  desirable  sales  type  of 
occupations,  there  is  a  stronger  tendency  for  the  group  to  be 
differentiated.  The  Record  of  Interests  as  filled  out  by  the  sales- 
men and  engineers  contained  this  list  of  thirty  occupations  which 
were  to  be  checked  plus  (  +  )  or  minus  (< — )  according  to  the 
directions  shown  with  them.  The  number  of  plus  and  minus 
checks  made  by  each  group  for  each  occupation  are  shown  in 
Table  VIII.  The  same  facts  are  shown  in  graphic  form  in  Dia- 
grams 7,  8,  9. 


Table  VIII 

Choices  of  Occupations  by  Insurance  Salesmen,  Sales  Engineers  and 

Design  Engineers 


Insurance  Salesmen 


Occupation 


1st  Class 


Actor    24  25 

Architect    28  19 

Auto  repairman    19  25 

Auto   salesman    52  8 

Bank  cashier    28  21 

Carpenter 14  23 

Captain  of  a  ship 22  23 

Chemist    17  29 

Detective     23  31 

Draftsman     16  36 

Editor  of  magazine  30  16 

Hardware   sales    14  36 

Hotel  keeper    19  25 

U.  S.  Astronomer   9  28 

Lawyer    37  10 

Locksmith     1  44 

Locomotive  engineer   20  19 

Machinist    14  26 

Newspaper   reporter    38  8 

Pattern-maker     5  39 

Policeman      1  50 

Private   secretary    38  9 

Purchasing  agent    46  8 

Real  estate  agent   42  8 

Research  worker  in  physics  12  27 

Sculptor     14  31 

Statistician    13  32 

Stockbroker    35  8 

Toolmaker    3  3 1 

Watchmaker     2  39 
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Diagram  7.  Number  of  Choices  Showing  Preferences  for  (+),  or 
Rejection  of  (— )  Occupations  by  Insurance  Salesmen.  Number  of 
men  represented,  124.  1 
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Diagram  8.  Percentages  of  Choices  Showing  Preference  for 
(-f-),  or  Rejection  of  ( — )  Occupations  by  Design  Engineers. 
Number  of  men  represented,  28. 
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Diagram  o.  Percentages  of  Choices  Showing  Preference  for 
(+),  or  Rejection  of  (— )  Occupations  by  Sales  Engineers. 
Number  of  men  represented,  50. 
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From  the  list  of  thirty  occupations,  twenty  occupations  were 
later  chosen,  because  they  had  given  the  best  results  for  differ- 
entiating; and  a  new  form  was  prepared  as  reproduced  on  page 
2)6.  The  student  engineers  are  directed  to  check  ten  of 
them  with  a  plus  (  +  )  as  the  more  desirable,  and  to  check  ten 
with  a  minus  ( — )  as  the  less  desirable.  The  twenty  occupa- 
tions classified  as  those  chosen  by  an  engineering  type  of  man 
and  those  chosen  by  a  sales  type  of  man  are  as  follows : 


Occupations  Chosen  by 
Engineering  Type 
Architect 

Automobile  repairman 
Carpenter 
Draftsman 

U.  S.  Government  astronomer 
Machinist 
Pattern-maker 

Research  worker  in  physics 
Toolmaker 
Watchmaker 


Occupations  Chosen  by 
Salesman  Type 

Automobile  salesman 

Bank  cashier 

Editor  of  popular  magazine 

Hotel  keeper   or  owner 

Lawyer 

Newspaper  reporter 

Private  secretary 

Purchasing  agent 

Real  estate  agent 

Stockbroker 


The  relative  percentage  of  the  sales  or  engineering  type  of 
occupation  chosen  by  a  man  indicates  whether  he  is  a  sales  type 
or  an  engineering  type  of  man.  The  scoring  method  explained 
more  fully  is  as  follows :  A  stencil  in  the  form  of  a  cardboard 
with  perforations  or  slots  allowing  only  the  engineering  type  of 
occupations  to  be  visible  is  placed  over  the  list  of  occupations 
which  have  been  checked.  The  number  of  plus  marks  is  counted 
and  recorded  in  the  margin.  The  number  of  minus  marks  is 
also  recorded.  Then  this  stencil  is  removed  and  another  stencil 
is  placed  over  the  list,  allowing  only  the  sales  type  of  occupation 
to  be  visible;  and  then  the  number  of  plus  and  minus  marks  is 
recorded.  The  number  of  plus  marks  before  engineering  occu- 
pations is  added  to  the  number  of  minus  marks  before  sales  occu- 
pations in  order  to  get  the  number  of  checks  in  favor  of  en- 
gineering occupations.  The  number  of  minus  marks  before  en- 
gineering occupations  is  added  to  the  number  of  plus  marks  before 
sales  type  of  occupations  to  get  the  number  of  checks  in  favor  of 
sales  occupations.  Finally  the  number  of  marks  in  favor  of 
sales  occupations  is  divided  by  the  total  number  of  check  marks 
to  get  the  percentage  of  marks  in  favor  of  sales  occupations. 
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It  would  not  be  necessary  to  count  both  the  plus  and  minus  marks 
if  all  the  subjects  checked  all  the  occupations ;  but  this  method  of 
scoring  can  be  used  even  though  not  all  the  occupations  are 
checked. 

The  Record  of  Interests  papers  of  the  salesmen  and  engineers 
were  scored  according  to  the  above  method ;  and  the  results  with 
only  the  twenty  selected  occupations  being  considered  are  pre- 
sented here.     Diagram  10  shows  the  percentages  of  sales  occupa- 
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Diagram  io.  Percentages  of  Choices  Showing  Preference  for  Occupations 
requiring  Sales  Type  of  Person. 

Considering  the  diagram  as  a  four- fold  table,  the  interest  test  (Record  of 
Interests)  places  correctly  eighty-two  per  cent  of  the  engineers,     ru  =  .84. 
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tions  chosen  by  each  group.  The  percentages  of  sales  occupa- 
tions chosen  are  shown  along  the  base  line  or  x-axis.  The  height 
of  the  columns  represents  the  number  of  men  choosing  the  par- 
ticular percentages  of  sales  occupations.  For  the  insurance  sales- 
men the  percentages  of  sales  occupations  chosen  were  73  per 
cent  for  the  first  class  and  80.6  per  cent  for  the  second  class.  For 
the  sales  engineers  the  preference  for  sales  occupations  was  67.7 
per  cent.  For  the  design  engineers  the  preference  for  sales  occu- 
pations was  only  30.0  per  cent. 

This  section,  "Choice  of  Other  Occupations,"  is  one  of  the 
most  reliable  sections  of  the  Record  of  Interests;  and  no  subject 
ever  failed  to  fill  it  out.  The  men  take  an  interest  in  choosing  the 
occupations  and  expressing  their  preference.  The  engineers  and 
the  salesmen  showed  a  definite  tendency  to  like  or  be  interested 
in  occupations  which  in  nature  of  work  were  similar  to  those 
they  were  already  following.  The  kind  of  occupations  which  they 
thought  would  give  them  the  greatest  satisfaction  to  follow  was 
a  significant  criterion  of  the  kind  of  work  in  which  they  could 
be  and  were  already  successful. 

By  computing  for  each  engineer  the  percentage  which  his 
choices  of  occupations  of  a  sales  nature  bore  to  his  total  number 
of  choices,  a  definite  measure  of  his  sales  engineering  interests 
as  opposed  to  design  engineering  interest  was  obtained.  I  as- 
sumed that  if  more  than  fifty  per  cent  of  a  man's  choices  were 
occupations  requiring  a  sales  type  of  person  for  success  in  them, 
that  man  was  a  sales  type  of  person;  and  if  more  than  fifty  per 
sent  of  the  man's  choices  were  occupations  requiring  a  design 
engineering  type  of  person,  that  man  was  a  design  engineering 
type  of  person.  By  this  measure,  78  per  cent  of  the  sales  engin- 
eers were  of  a  sales  type;  and  82  per  cent  of  the  design  en- 
gineers were  of  an  engineering  type.  Or  assuming  that  we  did 
not  know  the  actual  occupations  of  the  engineers,  those  en- 
gineers which  this  test  of  interests  alone  would  select  for  sales 
engineering,  would  be  89  per  cent  correctly  placed  or  classi- 
fied; and  the  men  which  this  test  of  interest  selected  for  design 
engineering  would  be  68  per  cent  correctly  placed  or  classified. 

III.     Avocational  Interests. — Questions  1,  2,  3,  and  4  did  not 


& 
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show  any  special  differentiation  except  as  they  showed  that  the 
sales  type  of  man  was  more  interested  in  the  social  sports  while 
the  engineering  type  often  enjoyed  photography  or  some  pro- 
fessional or  technical  hobby.  Question  5  showed  that  there  was 
a  definite  tendency  for  the  sales  type  of  man  to  take  more  active 
part  in  extra  academic  activities,  (See  Table  VII).  51  per  cent 
of  the  insurance  salesmen  of  the  first  class,  and  56  per  cent  of  the 
second  class,  47  per  cent  of  the  sales  engineers,  and  32  per  cent 
of  the  design  engineers  had  taken  part  in  student  debating.  45 
per  cent  of  the  first  class  and  48  per  cent  of  the  second  class  of 
insurance  salesmen,  27  per  cent  of  the  sales  engineers,  and  16 
per  cent  of  the  design  engineers  had  taken  part  in  dramatics. 
40  per  cent  of  the  first  class  and  52  per  cent  of  the  second  class 
of  insurance  salesmen,  33  per  cent  of  the  sales  engineers  and 
36  per  cent  of  the  design  engineers  had  taken  part  on  a  student 
paper  or  annual  book. 

IV.  Social  Interests. — The  answer  to  the  questions  on  social 
interests  were  not  significant  in  their  nature  in  differentiating 
the  men,  except  as  they  were  a  definite  means  of  showing 
whether  or  not  the  man  has  social  interests  and  is  congenial.  In 
answer  to  question  5,  60  per  cent  of  the  first  class  and  67  per 
cent  of  the  second  class  of  insurance  salesmen,  yy  per  cent  of 
the  sales  engineers,  and  yy  per  cent  of  the  design  engineers  pre- 
ferred a  roommate.  The  other  results  of  this  section  as  a  whole 
were  consistent  with  the  facts  of  differentiation  shown  in  the 
other  sections. 

V.  Interests  of  Relatives. — The  interests  of  the  relatives  as 
shown  by  their  occupations  indicated  a  nature  definitely  similar 
to  the  choice  made  by  the  man  considered,  but  there  were  excep- 
tions, so  that  the  choice  of  occupations  made  by  relatives  could 
not  be  used  as  a  criterion  of  the  proper  choice  for  the  man,  ex- 
cept as  it  might  be  consistent  with  and  strengthen  the  judgment 
of  a  man's  interests  as  based  upon  the  other  facts  of  the  total 
Record  of  Interests. 

VI.  Technical  Interests  in  the  Westinghonse  Organization 
and  Its  Opportunities. — Here  the  man  is  asked  for  a  definite 
statement  of  his  particular  interests  in  the  Westinghouse  Com- 
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pany.  The  results  obtained  in  this  section  agree  almost  perfectly 
with  the  choice  of  kind  of  occupation  indicated  in  the  other  sec- 
tions. 

IX.     Application  of  Psychological  Tests 

to  the  Problem 

i.     The  Nature  of  Mental  Ability  and  Its  Problems 

for  Tests 
By  psychological  tests,  we  mean  those  tests  which  are  con- 
sciously based  upon  psychological  principles  and  are  for  measur- 
ing a  certain  mental  process  or  processes.  Often  such  a  test  is 
intended  to  measure  general  intelligence.  By  general  intelligence 
is  meant  not  general  information,  but  what  is  synonymously 
called  menal  alertness,  mental  capacity,  or  innate  mental  ability. 
In  general,  psychologists  accept  the  theory  that  general  intelli- 
gence is  composed  of  a  group  of  mental  abilities  or  factors. 
Practically  the  same  conception  is  held  by  the  laymen ;  for  when 
a  layman  judges  or  makes  an  estimate  of  another  person's  gen- 
eral intelligence,  he  considers  that  person's  particular  capacities 
and  abilities  all  together,  and  approximates  a  sort  of  average  of 
them  which  is  compared  with  the  averages  of  other  people's 
capacities  and  abilities.  However,  it  is  recognized  that,  except  in 
the  sense  of  a  very  broad  general  intelligence,  people  are  indi- 
vidually different  in  that  the  general  intelligence  of  one  is  com- 
posed of  particular  capacities  and  abilities  different  in  degrees 
of  perfection  from  those  in  the  group  of  particular  abilities  com- 
posing another  person's  general  intelligence.  That  is,  it  may  be 
said  that  there  are  different  kinds  of  intelligences.  These  different 
kinds  of  intelligences  are  largely  due  to  particular  interests  direct- 
ing the  whole  general  intelligence  in  certain  directions;  but  on 
the  other  hand,  the  interests  are  given  substantial  basis  by  par- 
ticular abilities  which  make  activity  in  a  particular  line  easy  and 
interesting.  In  this  way,  what  is  practically  considered  as  a 
kind  of  intelligence  directed  by  particular  interests,  is  a  group 
of  special  abilities  developed  and  integrated  so  as  to  be  of  prac- 
tical use  in  the  life  of  the  individual.  If  general  intelligence  or 
any  kind  of  intelligence  is  analyzed  very  minutely,  it  is  found 
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to  consist  of  very  particular  abilities,  which  are  neural  bonds  be- 
tween a  particular  situation  and  a  particular  response.  Certain 
ones  of  these  are  essential  for  certain  special  abilities.  If  these 
essential  very  particular  abilities  are  not  present,  the  individual 
is  not  considered  as  specially  qualified  for  the  particular  work. 
Other  abilities  or  all  of  them  together  as  general  intelligence, 
may  function  vicariously  in  accomplishing  to  a  lesser  degree 
what  is  done  by  a  special  ability;  but  ability  in  that  particular 
kind  of  work  is  limited  by  the  weakness  or  lack  of  the  ability 
peculiarly  essential  for  marked  success  in  it.  It  is  the  differences 
between  individuals  in  the  degree  to  which  they  possess  these 
special  abilities  that  differentiates  them  for  particular  lines  of 
work.  Such  a  special  ability  integrated  and  functioning  with 
other  abilities  is  a  kind  of  intelligence,  which,  if  tested,  would 
yield  a  measure  of  potentiality  for  success  in  a  particular  kind  of 
occupations  or  kind  of  work.  Such  a  test  may  be  a  special  ability 
test  or  it  may  be  a  trade  test. 

Success  in  some  kinds  of  work  may  require  not  a  special  ability 
but  a  certain  amount  of  general  intelligence,  which  is  con- 
sidered as  a  certain  potentiality  in  all  of  the  large  group  of  abili- 
ties usually  functioning  in  the  many  activities  carried  on  to  a 
more  or  less  extent  by  all  persons.  For  an  occupation  in  which 
greater  general  intelligence  means  so  much  greater  capacity  for 
success,  persons  could  be  differentiated  by  a  general  intelligence 
test.  Such  a  test  measures  the  abilities  composing  and  therefore 
correlating  with  the  average  of  the  group  of  abilities  called  gen- 
eral intelligence. 

These  hypotheses  are  explained  here  as  being  the  most  per- 
tinent to  the  problem  being  studied.  They  are  stated  in  the  form 
of  hypotheses,  not  as  a  solution  to  the  problem,  but  as  a  restate- 
ment of  the  psychological  questions  raised  before  in  the  Introduc- 
tion. The  method  for  attempting  to  answer  the  psychological 
questions,  was  to  work  in  accordance  with  these  hypotheses. 
First,  a  general  intelligence  test  was  given  to  ninety-four  grad- 
uate student  engineers.  Later,  a  test  was  devised  which  was  in- 
tended to  test  special  ability  for  particular  kinds  of  work. 
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2.  The  Results  of  a  General  Intelligence  Test 
The  general  intelligence  test  used  was  the  Personnel  Bureau 
Test  VI.,  which  is  a  modified  form  of  the  Army  Alpha  general 
intelligence  test.  This  test  was  given  to  ninety-four  graduate 
engineers  who  were  students  in  the  Educational  Department  of 
the  Westinghouse  Electric  and  Manufacturing  Company.  Most 
of  these  men  had  been  in  the  Educational  Department  from  six 
months  to  a  year.  The  men  were  already  classified  according 
to  the  kind  of  work  to  which  they  were  to  be  assigned.  The 
average  of  the  test  scores  for  each  group  is  shown  in  Table  IX. 

Table  IX. 

Averages  of  General  Intelligence  Test  Scores  for  Each  Group  of  Graduate 

Student  Engineers. 

Estimated  Rank  in 
Mean  Median  Intelligence,  Corn- 

Group  Score  Score  mittee's  rating 

Design  engineers   154  :5°  J2-95 

Operating,  Service,  Works  140  144  84.2 

General  engineers    137  135  59-9 

Sales   engineers    129  130  56.5 

The  test  scores  for  each  group  of  engineers  are  shown  graph- 
ically in  Diagram  11,  in  which  the  height  of  each  column  indi- 
cates the  number  of  men  making  any  particular  score  shown  on 
the  horizontal  axis.  Although  it  has  not  been  the  intention  to 
select  the  men  with  greater  mental  ability  for  any  particular 
line  of  work,  the  results  of  the  test  indicate  that  there  has  been 
a  tendency  to  select  the  men  with  greater  mental  ability  for 
engineering,  particularly  design  engineering.  This  agrees  with 
the  results  of  the  study  of  the  technical  school  grades.  How- 
ever, the  difference  is  not  sufficient  to  differentiate  the  men,  ex- 
cept that  a  critical  score  might  be  established,  below  which  a 
man  could  not  be  accepted  for  design  engineering.  The  cor- 
relations between  this  test  and  other  tests  and  ratings  are  shown 
in  Table  X. 

On  examining  the  cases  in  which  the  test  scores  do  not  agree 
with  estimated  intelligence,  we  find  that  there  seem  to  be  certain 
tendencies  causing  the  disagreement.  A  table  of  the  cases  in 
which  there  is  the  greatest  disagreement,  and  those  in  which  there 


52 


BRUCE  V.  MOORE 


Scores  in  Bureau  Test  VI 
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Diagram  n.    Scores  in  Bureau  Test  VI  of  Graduate  Engineers  Classified 
by  Occupations. 


Table  X. 

Correlations  of  Bureau  Test  VI   Scores  with  Other  Criteria. 

Number  Correlation 

Criteria   Correlated  with  Test  VI.                           of  Cases  (r) 

Estimated   intelligence,   committee's    rating 74  +46 

Composite  of  technical  school  grades  56  -^--37 

Technical  school  grades  in  languages   43  .0 

Technical  school  grades  in  mathematics   34  +-34 

Technical  school  grades  in  shop   24  +-07 

Technical  school  grades  in  engineering   53  -\-.2& 

Technical  school  grades  in  academic  subjects  35  .0 

Company  shop  ratings   58  .0 

Ratings  at  employment  interview  18  +-58 

Rating  in  intelligence  at  employment  interview  18  +-57 

Bureau  Test  10,  Part  I  (Insurance  salesmen  Class  1)..  64  +-64 

Bureau  Test  10,  Part  I  (Insurance  salesmen  Class  2)..  58  +-91 

Bureau  Test  10,  Part  I  (Engineers)    18  +.69 

Bureau  Test  10,  Part  II  (Engineers)   18  +.38 
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is  the  greatest  agreement  is  presented  for  study  and  comparison 
of  those  cases.    See  Table  XI. 


Table  XI. 

Cases  in  Which  Test  Scores  Do  Not  Correlate  with  Estimated  Intelligence 
and  Cases  in  Which  the  Correlation  Is  High. 


Disagreement:    High  Rank  by  Test;  Low  Rank  by  Committee 


Number 
of  Case 


2 

3 

4 
5 
6 


Rank  by  Test 


5/2 
12 

18 

21 
12 
10 


Estimated 
Intelligence  Rank 

6i 

57 
63 
62 

49 
39 


Assignment 
Engineering 

Sales 

Engineering 

Operating 

Engineering 

Sales 


Disagreement 


7 
8 

9 
10 


11 
12 
13 


14 
15 

16 

17 
18 

19 


Low  Rank  by  Test;   High  Rank  by  Committee 

56/  13  Sales 

52  16  Sales 

52  17  Sales 

61  26  Engineering 


Agreement : 

65 
63 
48 


Both  below  Average 
66 

64 

48 


Agreement :     Both  above  Average 
1  1V2 


w 

6 

sy2 

9 

12 

12 

24 

25 

Sales 
Sales 
Sales 


Engineering, 

Design 
Engineering, 

Design 
Engineering 
Engineering 
Engineering 
Engineering 


In  the  group  of  those  who  received  a  high  rank  by  the  test, 
but  a  low  rank  by  the  committee,  it  is  seen  that  most  of  the  men 
in  this  group  are  engineers  as  contrasted  with  salesmen.  These 
men  tend  to  be  more  of  the  profound  intellectual  type  as  con- 
trasted with  the  bright  attractive  type.  For  this  reason  their 
pure  reasoning  or  intellectual  ability  tends  to  be  underestimated. 
It  is  also  interesting  to  note  that  in  the  case  of  greatest  disagree- 
ment, that  of  No.  1,  the  engineer  was  rated  on  the  rating  scale 
at  time  of  employment,  and  he  received  in  physical  qualities  only 
12  points  out  of  a  possible  20,  in  personal  qualities  only  12  points 
out  of  a  possible  20,  but  in  intelligence  he  received  20  points,  the 
highest  possible  rating. 
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Of  the  graduate  engineers  who  received  a  low  rank  by  the 
test  but  a  high  rank  by  the  committee,  three  are  assigned  as 
salesmen,  and  but  one  as  an  engineer.  It  is  seen  that  there  is  a 
tendency  for  the  man  with  high  intelligence  but  comparative  lack 
of  highly-rated  appearance  and  personal  qualities  to  be  under- 
estimated; but  the  man  with  good  or  pleasing-  appearance  and 
personal  qualities  is  over-estimated  in  intelligence.  This  is  the 
usual  tendency  often  found,  because  we  tend  to  consider  other 
qualities  which  are  evident,  but  which,  although  valuable  traits, 
are  not  intelligence  as  measured  by  the  test.  Of  those  cases  in 
which  there  was  perfect  or  most  nearly  perfect  agreement,  all 
those  ranking  above  average  were  engineers,  and  all  those  ranking 
below  average  were  assigned  to  sales  work.  It  seemed  very 
probable  that  in  selecting  men  for  engineering,  particularly  de- 
sign engineering,  intelligence  had  been  the  chief  qualification  con- 
sidered ;  but  in  choosing  men  for  sales  engineering,  other  qualifi- 
cations had  been  given  much  weight. 

3.     Construction  and  Evaluation  of  a  Special  Test  for 

Differentiating  Graduate  Student  Engineers  * 

(a)  Hypothesis  of  the  Test.- — As  was  stated  in  Section  II,  the 
problem  of  this  research  was  to  discover  or  devise  a  method  and 
means  for  selecting  from  the  whole  group  of  graduate  student 
engineers  those  who  would  give  their  services  most  efficiently  as 
sales  engineers.  The  general  intelligence  test  did  not  adequately 
separate  the  sales  engineers  from  the  other  engineers.  Evidently 
the  difference  between  sales  engineers  and  other  engineers  is  a 
qualitative  one  more  than  it  is  a  quantitative  one  in  intelligence. 
Moreover,  the  difference  is  a  relative  one ;  for  the  purpose  is  not 
so  much  to  pick  a  certain  standard  of  sales  engineers  as  it  is  to 
pick  a  certain  number,  about  half  of  the  group,  who  will  be  the 
best  selection  from  that  group  for  developing  into  sales  en- 
gineers. For  measuring  this  qualitative  difference  a  general 
intelligence  test  could  not  be  used.  Moreover,  any  general  intel- 
ligence test  available  was  too  easy  for  this  group  of  men,  because 
they  all  made  high  scores  and  were  not  sufficiently  differentiated 
by  it.     Finally,  the  content  or  subject-matter  of  existing  intelli- 


PERSONNEL  SELECTION  OF  GRADUATE  ENGINEERS         55 

gence  tests  was  not  relevant  to  the  engineering  profession,  and 
therefore  did  not  interest  or  appeal  to  engineers  as  a  test  might 
be  made  to  do.  For  these  reasons,  another  test  was  devised  to 
distinguish  two  different  kinds  of  intelligence. 

The  theory  upon  which  the  test  was  constructed  is  that,  of 
this  group,  the  sales  engineers  on  the  one  hand,  and  the  design 
engineers  on  the  other  hand,  are  relatively  at  opposite  extremes 
in  the  kind  of  intelligence  possessed.  The  sales  engineer  as  a  re- 
sult of  his  innate  particular  abilities,  his  environment,  and  his 
training,  has  interests  and  abilities  that  make  for  an  intelligence 
better  fitted  for  the  work  of  a  sales  engineer.  He  has  developed 
habits  of  thinking  and  attitudes  toward  other  people  which  are 
more  like  those  of  a  salesman ;  and  his  interests  and  apperceptions 
have  led  him  to  pick  up  and  retain  more  of  the  information,  par- 
ticularly of  a  business,  social,  and  human  interest  nature,  which 
enables  him  to  fit  into  the  work  of  a  sales  engineer  better  than  the 
design  engineer  type  would. 

The  design  engineer  type  has  innate  abilities  and  interests 
which  with  environmental  influences,  experiences,  and  training, 
have  developed  into  an  intelligence  better  fitted  for  the  work  of 
a  design  engineer.  He  worked  intensively  at  his  technical  prob- 
lems in  college,  and  has  developed  habits  of  thinking  and  atti- 
tudes which  are  more  compatible  with  the  work  of  a  design  en- 
gineer. It  was  with  these  subtle  differences  between  the  sales 
engineers  and  design  engineers  in  mind,  that  the  test  was  devised. 

(b)  Directions  for  the  Test. — The  nature  of  the  test,  being 
composed  partly  of  questions  of  a  technical  nature  as  well  as  of 
general  information,  makes  it  inadvisable  to  publish  it.  However, 
we  publish  here  the  directions  just  as  they  are  given  with  the 
test.  The  directions  include  samples  which  indicate  the  nature  of 
the  various  kinds  of  problems  in  the  test.  These  samples  are  to 
explain  the  test  so  that  the  subject  will  understand  clearly  what  he 
is  to  do  with  each  kind  of  problem ;  therefore,  they  are  simple  and 
easier  than  the  problems  and  questions  in  the  actual  test.  Never- 
theless, they  sufficiently  indicate  the  nature  of  this  test  for  the 
purposes  of  this  report. 
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TEST  10 

Test  for  Graduate  Student  Engineers 

Prepared  by  B.  V.  Moore,  Research  Fellow,  Expressly  for  the  Westinghouse 
Electric  &  Manufacturing  Company,  East  Pittsburgh,  Pennsylvania. 

Part  I 

To  the  Westinghouse  Student  Engineer: 

This  is  a  test  to  help  you  place  yourself  in  the  Westinghouse  Company 
where  you  can  be  most  successful.  The  results  of  the  test  will  be  held 
strictly  confidential.  You  are  asked  to  do  your  best  with  the  test  so 
that  the  Company  will  be  in  a  position  to  give  you  the  greatest  possible 
opportunity.  After  the  test  has  been  scored,  you  may  receive  your  score 
individually;  and  you  may  discuss  the  results  confidentially  with  the 
executives  of  the  Educational  Department. 

Fill  in  the  blanks  as  indicated. 

Name 

Place   of   Birth 

If  not  born  in  U.  S.,  how  many  years  have  you  been  in  U.  S.? 

Kind  of  Engineering  for  which  you  are  trained 

(Eletrical,  Mechanical,  etc.) 

Date  

Now  read  the  Directions  on  the  next  two  pages  very  carefully,  but  do 
not  turn  to  page  4  until  told  to  do  so. 

DIRECTIONS 

Inside  this  booklet  you  will  find  a  lot  of  things  to  do.  Samples  of  all 
the  different  kinds  of  things  to  be  done  are  given  below  with  directions 
for  doing  each  one.  Follow  the  directions  carefully,  and  pay  close  atten- 
tion to  the  examples  so  that  you  learn  to  do  each  thing  correctly. 

(a)  People  hear  with  the  eyes        ears        nose        mouth 

Jn  such  sentences,  one  of  the  last  four  words  will  make  the 
sentence  a  true  statement  of  fact.     Underline  the  right  word,  thus: 
People  hear  with  the    eyes        ears        nose        mouth 

Another  example  of  the  same  thing  correctly  marked,  is: 
France  is  in        Europe         Asia        Africa        Australia 

(b)  locomotive— train    ::   horse—        hub        buggy        car        baggage 

The  first  word,  "locomotive"  is  related  to  the  second  word, 
"train,"  in  the  same  way  as  the  third  word,  "horse,"  is  related  to 
one  of  the  words  following  it.  You  are  to  underline  that  word 
which  is  related  to  the  third  word  in  the  same  way  as  the  first  two 
words  are  related  to  each  other.  In  this  example,  "locomotive" 
is  related  to  "train"  as  "horse"  is  related  to  "buggy";  for  a  locomo- 
tive pulls  a  train,  and  a  horse  pulls  a  buggy.  Therefore,  "buggy" 
should  be  underlined,  thus: 
locomotive— train  ::  horse—        hub        buggy        car        baggage 

Another  example  of  the  same  thing  correctly  marked,  is: 
woman — beautiful   ::  man —         girl         old         mother        handsome 

Handsome    is   underlined   because   a   man   is   described    as   hand- 
some in  the  same  way  as  a  woman  is  described  as  beautiful. 
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(c)  The   concert   was         (delicious        lovely        delightful) 

In  such  sentences,  only  one  of  the  words  or  phrases  in  paren- 
thesis can  be  used  correctly.  You  are  to  underline  the  correct 
word,  thus: 

The  concert  was         (delicious        lovely        delightful) 

Another  example  of  the  same  thing  correctly  marked,  is: 
He         (isn't        doesn't        don't)         do   that   kind  of   work. 

(d)  little        same  as        large        child        big        small 

In  this  kind  of  problem,  there  is  always  just  one  word  in  the 
group  of  four  words  which  has  the  same  meaning  as  the  single 
word  given  first.     In  this  example,  "little"  has  the  same  meaning  as 
"small."  Therefore,  "small"  should  be  underlined,  thus : 
little         same  as        large        child        big        small 

Another  example  of  the  same  thing  correctly  marked,  is: 
talk        same  as        story        truth        speak        hear 

(e)  soft        opposite  of        brittle        hard        yielding        rough 

In  this  kind  of  problem,  there  is  always  just  one  word  in  the 
group  of  four  words  which  has  exactly  the  opposite  meaning  of  the 
word  given  first.  In  this  example,  "soft"  has  the  opposite 
meaning  of  "hard."  Therefore,  "hard"  should  be  underlined,  thus: 
soft        opposite   of        brittle        hard        yielding        rough 

Another  example  of  the  same  thing  correctly  marked,  is: 
encourage    opposite    of    courage  inconvenience  discourage  inspire 

(f)  Don't  put  all  your  eggs  in  one  basket. 

The  mouse  that  has  only  one  hole  is  soon  caught. 

....Catch  the  bear  before  you  sell  his  skin. 
....The  proof  of  the  pudding  is  in  the  eating. 

In  such  groups  of  statements,  there  is  always  just  one  which 
gives  essentially  the  same  meaning  as  the  first  one.  Put  a  cross 
before  the  statement  which  means  the  same  as  the  first  statement, 
thus: 

Don't  put  all  your  eggs  in  one  basket. 
.X  .The  mouse  that  has  only  one  hole  is  soon  caught. 
....Catch  the  bear  before  you  sell  his  skin. 
....The  proof  of  the  pudding  is  in  the  eating. 

Another  example  of  the  same  thing  correctly  marked,  is: 
Long  absent,  soon  forgotten. 
....Absence  makes  the  heart  grow  fonder. 
. .  ..Distance  lends  enchantment  to  the  view. 
.X  .Far  from  eyes,  far  from  heart. 

When  the  signal  is  given,  you  are  to  turn  to  page  4  and  do  these 
things  as  you  have  learned  to  do  them  here.  Do  not  ask  questions.  If 
you  forget  how  to  do  any  of  them,  you  may  turn  back  to  these  pages, 
but  to  do  so  unnecessarily  will  waste  your  time. 

Work  rapidly.  You  will  be  allowed  twenty  minutes.  You  may  not 
be  able  to  get  through  in  that  time;  but  do  as  much  as  possible. 

Mark  every  question.  If  you  are  not  sure  about  your  answer  to 
something,  guess  at  it,  and  go  on  to  the  next  question.  There  are  no 
catch  questions. 

When  the  signal  is  given,  begin  at  the  top  of  the  next  page,  and 
work  through  the  remainder  of  the  booklet,  without  skipping  about. 

Do  not  turn  over  to  the  next  page  until  you  are  told  to  do  so. 


58  BRUCE  V.  MOORE 

Part  II 

On  the  following  pages  are  some  more  things  for  you  to  do. 
Samples  of  the  three  different  kinds  of  things  to  be  done  are  given  below 
with  directions  for  doing  each  one.  Follow  the  directions  carefully,  and 
pay  close  attention  to  the  examples  so  that  you  learn  how  to  do  each  thing 
correctly. 

(a)  silver        copper        glass        aluminum        gold 

In  such  groups  of  live  words,  there  are  four  words  which  rep- 
resent things  or  ideas  that  can  be  thought  of  as  similar  in  a  certain 
way.  That  is,  they  can  be  classed  under  the  same  fundamental 
idea  or  conception.  One  of  the  five  words  represents  something 
that  is  not  like  the  other  four  words  in  the  way  that  they  are  all  like 
each  other;  and  therefore,  it  cannot  be  classed  with  them  under  the 
same  conception  or  category.  You  are  to  discover  the  fundamental 
idea  of  the  classification,  and  underline  the  word  that  is  not 
included.  In  the  example  above,  all  the  materials  are  conductors  of 
electricity,  except  "glass,"  which  is  relatively  a  non-conductor. 
Therefore  "glass"  should  be  underlined,  thus: 
silver        copper        glass        aluminum        gold 

Another  example  of  the  same  thing  correctly  marked,  is : 
shunt  motor        transformer        telegraph .  sounder 

synchronous  alternater  exciter        electroplating  apparatus 

Transformer  is  underlined  because  it  has  to  do  with  alternating 
current,  but  the  other  four  words  have  to  do  with  direct  current. 

(b)  Other    factors    remaining    constant,    the    electric    current    in    a    wire 

varies  inversely  as  the  resistance  of  the  wire. 

True        False  d 

You  are  to  decide  whether  such  statements  are  true  or  false,  and 
then  underline  the  appropriate  word,  True  or  False,  which  indi- 
cates the  nature  of  the  statement.  The  above  statement  is  true; 
therefore,  "True"  should  be  underlined,  thus: 

Other    factors    remaining    constant,    the    electric    current    in    a    wire 
varies  inversely  as  the  resistance  of  the  wire. 

True        False 

Another  example  of  the  same  thing  correctly  marked,  is: 
Laminated  armature  cores  are  used,  because  they  retain  magnetism 
better. 

True        False 

(c)  What  direct  current  at  no  volts  will  give  the  same  horsepower  as  a 

direct  current  of  5  amperes  at  220  volts? 

Answer amperes 

Such  problems  are  to  be  solved.  You  may  use  the  margin  of 
the  pages  for  figuring.  Write  the  answer  in  the  place  indicated, 
thus: 

What  direct  current  at  no  volts  will  give  the  same  horsepower  as  a 
direct  current  of  5  amperes  at  220  volts? 
Answer 10 amperes 

Another  example  correctly  solved,  is: 
A  direct  current  motor  uses  a  current  of  7.46  amperes  at  250  volts. 
A   Pony  brake  test   shows   that   the   motor   is   giving  2  horsepower. 

What  is  the  efficiency  of  the  motor? 

Answer. . .  .80%.. .. 
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When  the  signal  is  given,  you  are  to  turn  to  the  next  page  and 
do  these  things  as  you  have  learned  to  do  them  here.  Do  not  ask  ques- 
tions. If  you  forget  how  to  do  any  of  them,  you  may  turn  back  to 
these  pages,  but  to  do  so  unnecessarily  will  waste  your  time. 

Work  rapidly.  You  will  be  allowed  thirty  minutes.  You  may  not 
be  able  to  get  through  in  that  time,  but  do  as  much  as  possible. 

Mark  every  question.  If  you  are  not  sure  about  your  answer  to 
something,  guess  at  it,  and  go  on  to  the  next  thing.  There  are  no  catch 
questions. 

When  the  signal  is  given,  begin  at  the  top  of  the  next  page,  and 
work  through  the  remainder  of  the  booklet,  without  skipping  about. 

Do  not  turn  over  the  page  until  you  are  told  to  do  so. 

(c)  Construction  of  the  Test. — A  test  is  developed  and  im- 
proved much  as  a  new  machine  is  developed.  It  is  brought  nearer 
perfection,  and  the  final  efficiency  attained  by  eliminating  one 
difficulty  after  another  and  by  taking  advantage  of  one  pos- 
sibility for  improvement  after  another,  just  as  the  dynamo  was 
brought  nearer  perfection  from  a  cast  iron  bipolar  series-wound 
machine  to  the  present  efficient  machine  with  laminated  cores, 
commutating  poles,  commutating  pole-face  windings,  compound 
wound  field,  balance  coils,  etc.  This  test  was  developed  in  much 
the  same  way.  It  may  be  found  later  that  some  of  the  reasons 
for  making  the  test  as  it  is,  have  no  basis  of  fact;  but  the  inten- 
tion was  not  to  fail  to  take  advantage  of  any  possible  means  or 
device  for  differentiating  between  the  sales  engineer  and  the  de- 
sign engineer  type.  Therefore,  some  of  the  features  of  the  test 
may  seem  trivial  and  even  arbitrary ;  but  they  were  not  discarded, 
because  there  was  no  proof  of  their  futility. 

To  discuss  the  test  more  in  detail,  we  shall  begin  with  the  in- 
formation questions  of  Part  I.  (See  Directions,  a).  Of  the  one 
hundred  sixty  questions  in  Part  I,  forty,  or  one-fourth,  are  ques- 
tions of  information.  It  is  believed  that  the  sales  type  of  engineer 
has  been  interested  in  other  things  in  addition  to  his  books  and 
strictly  technical  training.  He  not  only  has  been  a  good  mixer 
in  college  and  has  taken  part  in  extra-academic  activities,  but  he 
also  has  been  interested  in  the  affairs  of  the  social,  economic,  busi- 
ness, and  work-a-day  world.  The  potential  salesman's  abilities, 
and  with  them  his  interests,  have  tended  to  select  and  retain  for 
him  those  items  of  information  which  are  peculiar  to  a  sales  type 
of  person  and  make  him  such  a  person.    A  definite  method  was 
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followed  in  order  to  have  the  questions  such  that  a  sales  type  of 
person  would  answer  best.  After  studying  the  occupational 
descriptions  of  the  various  lines  entered  by  sales  engineers,  a 
definite  prospective  customer  in  a  particular  industry  was 
imagined.  Supposing  that  I  was  sales  manager,  I  outlined  the 
kind  of  experiences  and  information  I  should  want  my  salesman 
going  to  that  customer  to  have.  After  doing  this  for  each  class 
of  customer  for  each  sales  department,  I  prepared  a  composite  list 
of  kinds  of  information  and  experience  which  I  should  expect  a 
successful  sales  type  of  engineer  to  have  accumulated  in  his 
early  life,  his  high  school  and  college  life,  his  social  and  fra- 
ternal life,  his  business  life,  his  sports,  amusements,  and  avoca- 
tions, and  his  reading.  With  this  list  before  me,  I  selected  from 
other  tests  and  formulated  new  questions  which  would  cover 
these  twenty  or  more  different  classes  of  information  and  inter- 
ests. 

The  second  type  of  problem  chosen  was  the  analogies  test. 
(See  Directions,  b).  Forty  items  of  this  type  were  included.  In 
studying  the  general  intelligence  test  papers,  Bureau  Test  VI, 
the  scores  of  the  sales  engineers  in  the  different  kinds  of  tests 
making  up  the  total  test  were  compared  with  the  scores  in  those 
tests  of  the  other  engineers  making  the  same  total  score.  It  was 
found  that  with  the  small  number  considered,  twenty-five  sales 
engineers  and  sixteen  others,  the  sales  engineers  did  better  in 
the  analogies  test,  although  their  total  score  on  the  whole  exam- 
ination was  the  same  as  the  other  engineers'  with  whom  they  were 
compared. 

The  next  three  tests  might  be  considered  or  classified  as 
vocabulary  tests,  but  they  are  intended  to  be  also  more  than  that. 
Recognizing  the  fact  that  the  salesman  must  be  able  to  express 
his  ideas  and  understand  what  others  are  trying  to  say,  an  at- 
tempt is  made  to  measure  not  merely  his  knowledge  of  words, 
but  also  his  sense  of  their  finer  meanings  and  implications  in 
certain  usages.  The  correct  answer  to  a  particular  test  item  of 
this  sort  often  cannot  be  made  according  to  any  dictionary  defini- 
tion or  any  rule  of  rhetoric.     It  depends  upon  usage  and  a  fine 
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sense  of  propriety.  To  answer  them  correctly  the  man  cannot 
stop  to  reason  out  the  answer,  or  he  will  be  lost;  but  he  must 
respond  as  he  feels  about  the  proper  sense  of  the  word.  It  is  an 
attempt  to  measure  the  ability  not  of  using  words  but  of  compre- 
hending the  meaning  of  other  people's  statements  and  of  getting 
other  people  to  undertsand  us  as  we  expect  them  to  understand 

us. 

The  proverbs  are  intended  to  measure  somewhat  the  same  thing 
as  the  preceding  types  of  tests  measure,  namely  ability  to  read 
and  understand  sentences.  In  addition,  it  is  considered  as  a  test 
of  the  ability  to  generalize  or  think  consistently.  It  measures 
the  ability  to  interpret  the  gist  of  a  complete  statement  rather  than 
the  meaning  of  words  or  phrases.  In  the  proverbs  test,  several 
possible  forms  were  considered.  I  thought  at  first  that  Professor 
L.  L.  Thurstone's  form,  a  prototype  with  four  other  sentences, 
would  be  used ;  but  this  was  abandoned  as  too  long  for  one  item. 
Then  I  thought  that  it  would  be  possible  to  score  Professor 
Thurstone's  form  as  two  items,  one  point  for  each  sentence  or 
phrase  checked  correctly.  But  considering  the  probabilities  in- 
volved, this  plan  was  dropped.  By  that  plan,  a  subject  in  select- 
ing the  first  of  the  two  sentences  to  be  checked,  has  a  two-out-of- 
four  or  fifty-fifty  chance.  If  he  is  a  capable  subject  he  gets  the 
first  one  right.  Then  in  checking  the  second  one  he  has  a  chance 
of  only  one  out  of  three;  and  thus  the  problem  is  made  more 
difficult.  However,  if  he  were  a  much  less  capable  subject,  the 
first  one  he  checks  might  very  possibly  be  a  wrong  one ;  but  he  has 
two  chances  out  of  three  of  selecting  the  right  one  in  checking  a 
right  sentence  as  the  second  one.  Thus  the  problem  is  made 
more  difficult  for  the  more  capable  and  less  difficult  for  the  less 
capable ;  and  there  is  a  tendency  to  eliminate  a  showing  of  differ- 
entiation in  capacity.  For  these  reasons,  the  proverbs  were  con- 
structed so  that  there  would  be  just  one  probability,  namely  one 
out  of  three  chances. 

Part  II.  is  apparently  a  test  of  technical  engineering  ability. 
However,  it  is  not  to  be  considered  a  trade  test.  It  would  be 
such  if  it  were  given  to  a  group  of  men,  some  of  whom  were 
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engineers  and  some  of  whom  were  not,  but  the  test  is  for  en- 
gineers only,  and  is  not  to  distinguish  engineers  from  people 
that  are  not  engineers.  The  technical  information  required  to 
pass  the  test  is  comparatively  easy  for  electrical  engineers ;  for 
the  test  was  designed  to  require  only  such  information  and  en- 
gineering ability  as  every  graduate  of  all  standard  schools  would 
have.  When  the  test  is  given  to  a  group  of  engineers,  it  is  essen- 
tially an  intelligence  test;  but  it  tests  a  certain  kind  of  intelligence. 
The  hypothesis  upon  which  it  is  based  is  that  the  man  who  is 
more  capable  as  an  engineer,  who  has  his  interests  in  the  pro- 
fession, and  who  has  held  himself  more  intensively  to  his  books, 
shop,  drafting  board,  and  mathematics,  is  the  design  engineer 
type,  and  is  the  kind  of  engineer  who  will  tend  to  pass  this  kind 
of  test  better  than  he  would  pass  Part  I.  It  is  expected  that  a 
pure  design  engineer  type  of  man  will  pass  such  an  intelligence  test 
better  if  the  vocabulary  and  content  is  that  with  which  he  is  ac- 
customed to  work  and  think  about. 

The  first  kind  of  problem  or  question  in  Part  II,  (a),  is,  as  the 
Directions  explain,  a  test  of  ability  to  deal  with  fundamental  con-  ., 

ceptions  found  in  mechanics,  physics,  or  engineering.  The  test 
is  intended  to  measure  the  subject's  alertness  in  dealing  with  or  in 
juggling  such  technical  ideas.  It  is  even  hoped  to  be  a  test  of 
association  or  originality  in  seeing  the  relations  involved  in  such 
fundamental  conceptions.  It  calls  for  analysis  and  generaliza- 
tion in  the  technical  field. 

The  second  type  of  question,  Part  II,  Directions  (b),  is  com- 
posed largely  of  questions  calling  for  technical  information;  but 
the  questions  are  so  stated  that  the  subject  must  read  them  care- 
fully and  think  straight  to  give  the  right  answer;  for  they  can- 
not be  answered  directly  from  general  principles  or  textbook 
information,  but  require  the  application  of  general  principles. 

The  third  type  of  problem,  Part  II,  Directions,  (c),  is  mani- 
festly a  straight  mathematical  engineering  problem.  The  prob- 
lems are  not  highly  technical,  but  require  straight  thinking  and 
accurate  computation.  Problems  in  mensuration,  mechanics, 
and  general  physics  are  included  as  well  as  problems  in  electrical 
engineering. 
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In  a  psychoiogical  test  with  a  time  limit,  it  seems  most  logical 
to  have  the  steps  or  items  to  be  solved,  of  equal  difficulty,  par- 
ticularly if  speed  is  considered  as  a  measurable  factor  of  the 
ability  to  be  tested.  Therefore,  if  the  items  cannot  be  made  of 
equal  difficulty,  it  does  not  seem  consistent  to  arrange  the  items 
in  the  order  of  their  difficulty,  putting  the  most  difficult  to  be 
solved  last.  This  tends  to  minimize  the  differentiating  process 
rather  than  make  the  most  of  it.  The  capable  subject  gets 
through  the  easier  items  first,  but  is  held  back  by  the  more  difficult 
material  ahead,  while  the  less  capable  subject  coming  through 
the  easier  material  then  tends  to  catch  up  with  the  capable  man. 
Moreover,  the  difficult  material  which  the  capable  person  might 
be  able  to  solve,  but  which  the  less  capable  man  never  could 
solve,  is  never  reached  by  either.  It  might  be  argued  that  the 
more  difficult  material  be  placed  first  and  the  easier  later,  so  that 
the  less  capable  subjects  would  have  to  pass  it  over  after  spend- 
ing a  little  time  on  it.  The  capable  man  would  get  credit  for  it 
and  also  for  much  more  easier  material,  which  would  make  his 
score  much  larger  than  that  received  by  the  less  capable  man. 
Thus  the  differentiation  would  be  greater.  For  these  reasons,  no 
especial  effort  was  made  to  rank  the  problems,  particularly  in 
Part  II,  in  order  of  difficulty;  but  some  attempt  was  made  to 
keep  the  difficulty  fairly  constant.  Of  course  the  first  few  items 
were  made  easier  so  that  all  subjects  could  get  the  correct  idea 
and  get  fairly  started. 

(d)  Standardization  of  the  Test. — In  the  process  of  develop- 
ing and  testing  the  test,  Part  I.  was  given  first  to  the  members  of 
the  seminar  of  the  Bureau  of  Personnel  Research,  composed  of 
two  persons  with  the  degree  of  Doctor  of  Philosophy  in  psycholo- 
gy, and  six  graduate  students.  The  number  of  the  item  which 
was  reached  by  each  subject  at  the  end  of  twenty  minutes,  was 
checked,  and  the  subjects  asked  to  complete  all  the  items  of  the 
test.  Also,  the  subjects  in  the  seminar  were  asked  to  criticize 
and  put  a  question  mark  after  each  item  that  was  ambiguous  or  of 
questionable  reliability.  Then  both  the  errors  and  the  criticisms 
were  tabulated  for  each  of  the  one  hundred  sixty  items.  After 
making  a  few  minor  changes  in  Part  I.  as  a  result  of  this  first 
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trial,  it  was  given  to  a  class  in  the  School  of  Insurance  Sales- 
manship. Sixty-one  men  and  three  women,  about  half  of  whom 
were  college  people,  were  in  this  group  taking  the  test.  Part  I. 
was  then  revised  a  second  time  and  given  to  fifty-five  men  and 
five  women  in  another  class  in  the  School  of  Insurance  Sales- 
manship. 

The  process  of  constructing  and  standardizing  Part  II.  was 
somewhat  different.  The  True-False  questions  were  taken  from 
Professor  E.  L.  Thorndike's  engineering  test.  The  problems 
were  taken  partly  from  Professor  L.  L.  Thurstone's  engineering 
tests,  partly  from  standard  engineering  textbooks,  and  part 
were  originally  devised  by  myself.  The  disparate  word  test  items, 
that  is,  the  groups  of  words  with  one  word  not  included  in  the 
fundamental  conception,  were  all  originally  devised  by  myself. 

For  standardizing  Part  II.,  it  was  studied  and  the  problems 
worked  by  two  engineers  of  the  Westinghouse  Electric  and 
Manufacturing  Company.  Then  the  problems  were  worked,  and 
the  whole  of  Part  II.  carefully  criticized  by  Professor  W.  R. 
Work  of  the  Department  of  Electrical  Engineering,   Carnegie  t 

Institute  of  Technology.  Then  both  Part  I.  and  Part  II.  were 
given  to  three  seniors  in  electrical  engineering  just  graduating 
from  the  Carnegie  Institute  of  Technology.  After  a  third  re- 
vision, the  entire  test,  Part  I.  and  Part  II.,  was  given  to  twenty- 
eight  design  engineers  and  two  general  engineers  in  the  Westing- 
house  Electric  and  Manufacturing  Company.  Later  the  entire 
test  was  given  to  fifty-nine  sales  engineers  in  the  same  company. 

The  specifications  for  selecting  the  Westinghouse  engineers  for 
the  test  were:  (i)  They  all  should  be  graduates  of  technical 
schools.  (2)  They  should  have  been  out  of  school  and  in  actual 
work  with  the  Company  not  more  than  five  years  and  not  less 
than  one  year.  (3)  The  design  engineers  should  be  beyond 
doubt  of  the  type  desired  in  design  engineering.  They  should 
have  proven  successful  and  should  be  satisfied  in  their  work. 
There  should  be  no  doubt  that  they  were  properly  placed.  (4) 
The  sales  engineers  should  be  beyond  doubt  of  the  type  desired 
in  the  sales  department.     They  should  have  proven  successful 
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and  should  be  satisfied  in  their  work.  There  should  be  no  doubt 
that  they  were  properly  placed  as  sales  engineers. 

With  these  specifications,  the  heads  of  the  engineering  depart- 
ments selected  the  design  engineers ;  and  the  whole  list  was  gone 
over  carefully  by  the  Chief  Engineer.  The  sales  engineers  were 
selected  in  a  similar  way  by  the  sales  managers.  A  few  of  the 
sales  engineers  had  been  out  of  school  more  than  five  years;  but  it 
is  believed  that  the  two  groups  were  composed  of  as  pure  type 
representatives  of  the  two  kinds  of  engineers  as  could  be  ob- 
tained. In  addition  to  helping  make  the  selection,  the  Chief 
Engineer  rated  the  design  engineers  on  their  ability  as  design 
engineers;  and  the  sales  managers  rated  the  sales  engineers  on 
their  ability  as  sales  engineers. 

(e)  Results  of  the  "Test  for  Graduate  Student  Engineers." — 
The  results  of  the  test  given  to  the  different  groups  can  be  sum- 
marized briefly.  Because  the  test  was  revised  after  giving  it  to 
the  first  class  in  the  School  of  Insurance  Salesmanship,  the  scores 
can  not  be  compared  with  the  scores  of  the  other  groups.  The 
insurance  salesmanship  classes  had  also  taken  Bureau  Test  VI., 
and  the  correlations  between  the  scores  of  that  test  and  those  of 
Part  I,  were,  r  =  +.64  for  the  first  class,  which  took  the  original 
form  of  the  test;  and  r  =  -\-  .92  for  the  second  class,  which  took 
the  revised  form  of  the  test.  Of  the  engineers  who  took  Bureau 
Test  VI,  18  also  took  the  Bureau  Test  10.  For  these  18,  the 
correlation  between  Test  VI  and  Part  I  of  Test  10  was  r  =  +.69; 
and  between  Test  VI  and  Part  II,  the  correlation  was  r  =  +.38. 
The  correlation  between  the  committee's  original  estimate  of  in- 
telligence and  Part  I  was  r  =  +49;  and  between  estimated  in- 
telligence and  Part  II,  the  correlation  was  r  =  -\-.76.  Evidently 
the  estimate  of  intelligence  was  influenced  by  the  student  engin- 
eer's manifestation  of  engineering  capacity. 

Considering  only  the  design  engineers,  Diagrams  12,  13,  and 
14  are  four-fold  tables  showing  the  relation  between  the  results 
of  Part  I  and  the  Chief  Engineer's  ratings,  between  Part  II  and 
these  ratings,  and  between  a  composite  of  Part  I  and  Part  II  and 
these  ratings.  The  composite  score  was  made  by  adding  the 
score  in  Part  I  to  the  score  in  Part  II  in  terms  of  standard  devia- 


Ratings   by  Chief   Engineer 

Lower 

Higher 

c 

•o 

4> 

I     1    1    1 
1 

1    1     1    1 
1     1     1    1 

1     1     1 

rt  ^a 

^< 

•S    ! 

t/>  c 

t)    to 

01j 

1     1    1     1 

1     1     1    1 

c/)£| 

1    1 

1     1     1 

S 

O 

u 

03 

Diagram  12.  Comparison  of 
Scores  in  Part  I  with  Chief  En- 
gineer's Rating  of  Design  Engi- 
neers,   ru  =  .22. 


c 

■9 

~£ 

f  £ 

I-   o 
03.0 

C 

'     c 

Ss 
& 

41 

pq 


Ratings   by   Chief   Engineer 
Lower  Higher 


1  1 

1    1    1    1 
1    1    1    1 
1    1    1    1 

MM 
1    1    1    1 
1 

MM 
1    1    1 

Diagram  13.  Comparison  of 
Scores  in  Part  II  with  Chief  En- 
gineer's Rating  of  Design  Engi- 
neers,   ru  =  .59. 


Ratings  by  Chief   Engineer 

(— 1 

Lower 

Higher 

t-<  c 

-  rt  1 

t)2 

§2 

1    1 

1   1   M 

MM 

tL  > 
m  ° 

TO  J2 

MM 

u 

8.1 

C/)T3 

Mil 

1    1    1   1 

V 

5S 

MM 

1    1    1 

O      & 

1 

O,    O 

E« 

Off) 

Ratings  by  Sales  Managers 
Below  Median      Above  Median 


U 


Diagram  14.  Comparison  of 
Scores  in  Composite  of  Part  I 
and  Part  II  with  Chief  Engineer's 
Ratings  of  Design  Engineers, 
ru  =  .66. 

Ratings  by  Sales  Managers 
Lower  Higher 


c 


■C  2 


O    aj 


MM 
1    1 

1   1    1   II    1 
1   II   1   II 
1   1    1   M    1 
Mill 

M   1    I    M 
M  II   M 
M  1    1   II 
II  II   M 

MM 
1    1 

Diagram  16.  Comparison  of 
Scores  in  Part  II  with  Sales 
Managers'  Ratings  of  Sales  En- 
gineers.   rn  =  -f  .81. 


•a 


v 

■>->  > 
M   2 


en   C 

ffl 


1  II  1 
1 

1  II  II  1 
1  1  II  1  1 

1  M    II    1 
1  1    II   ll 

1  1  II  1  1 
1  II  II  1 
1  1  II  1  1 
1  1  II  1  1 

1    II   1 

1 

Diagram  15.  Comparison  of 
Scores  in  Part  I  with  Sales  Man- 
agers' Ratings  of  Sales  Engineers. 
ru  =  +  .86 

Ratings  by  Sales  Managers 
Below  Median      Above  Median 


c 

£  > 

< 

tn 
U 

o  to 

s  & 

P  ~v 
off) 
U 


/Ml 
l    II   1 

1  1  1    1    II 
1  1  II    1    1 
1  I   M    M 
1  1    1 

1   1    II   II 
1   1    M   II 
1   1    1    1   1    1 
MM 

1    1    1    1 
/Ml 

Diagram  17.  Comparison  or 
Scores  in  Composite  of  Part  I 
and  Part  II  with  Sales  Man- 
agers' Ratings  of  Sales  Engi- 
neers,   ru  =  -f-  .66. 


PERSONNEL  SELECTION  OF  GRADUATE  ENGINEERS         67 

tions.  It  is  evident  that  Part  II,  the  technical  part  prepared  for 
design  engineers,  correlates  more  nearly  perfectly  with  the  rat- 
ings than  Part  I  does.  The  composite  score,  which  takes  into 
account  both  the  special  engineering  capacity  and  the  general  in- 
telligence or  mental  alertness  of  the  man,  correlates  a  little  more 
nearly  perfectly.  Diagrams  15,  16,  and  17  show  the  same 
relations  for  the  sales  engineers ;  that  is,  the  correlations  between 
the  test  and  the  ratings  made  by  the  sales  managers. 

Considering  again  only  the  design  engineers,  the  mean  of  the 
scores  in  Part  I  was  109.3;  and  the  mean  of  the  scores  in  Part 
II  was  37.0,  which  is  approximately  one-third  of  the  mean  for 
Part  I,  37.0  multiplied  by  three  being  111.0.  For  the  sales  en- 
gineers, the  mean  of  the  scores  in  Part  I  was  87.6;  and  the  mean 
of  the  scores  in  Part  II  was  22.7,  which  is  approximately  one- 
fourth  of  the  mean  for  Part  I,  22.7  multiplied  by  four  being 
90.8.  Thus  the  sales  engineers  do  relatively  better  in  Part  I  as 
compared  to  Part  II  than  the  design  engineers  do.  Comparing 
in  another  way,  the  design  engineers  do  relatively  better  in  Part 
II  as  compared  to  Part  I  than  the  sales  engineers  do;  for  the 
mean  of  their  scores  in  Part  II  is  more  than  one-third  of  the  mean 
of  the  scores  in  Part  I. 

Diagram  18  is  a  scatter  diagram  showing  the  relation  between 
Part  I.  and  Part  II.  for  the  scores  of  the  design  engineers  and 
the  sales  engineers.  Diagram  19  shows  the  same  relation  between 
the  percentiles  of  each  man  in  Part  I  and  Part  II.  The  circles 
(O)  represent  the  sales  engineers;  and  the  crosses  (X)  represent 
the  design  engineers.  One  purpose  of  the  test  is  to  distinguish 
sales  engineers  from  design  engineers.  According  to  the  hypo- 
thesis for  the  test,  the  men  who  did  well  in  both  parts  of  the 
test  and  appear  in  the  upper  right-hand  quadrant  of  the  diagram, 
are  mentally  capable  of  being  either  sales  engineers  or  design 
engineers  because  of  their  superior  general  all-around  intelligence, 
or  because  of  their  superior  intelligence  in  at  least  these  two  fields. 
The  men  who  did  poorly  in  Part  II.  but  very  good  in  Part  I, 
should  be  those  whose  kind  of  intelligence  fits  them  better  for 
the  work  of  a  sales  engineer.  The  larger  percentage  of  circles 
in  this  quadrant  substantiates  this  part  of  the  hypothesis.    Those 
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neers (o). 

who  did  poorly  in  Part  I.  but  very  good  in  Part  II.,  should  be 
those  whose  kind  of  intelligence  fits  them  better  for  the  work 
of  a  design  engineer.  The  larger  percentage  of  crosses  appear- 
ing in  this  quadrant  substantiates  this  part  of  the  hypothesis. 
Those  doing  poorly  in  both  Part  I.  and  Part  II.,  might  be  equally 
successful  in  either  line  of  work,  as  far  as  mental  ability  is  con- 
cerned; but  they  are  not  as  successful  as  any  of  the  others  might 
be  if  properly  placed  in  the  right  line  of  work.  As  a  whole  the 
hypothesis    is    substantiated    by    the    results.    There    are    some 
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Diagram  19.  Scatter  Diagram  Showing  Relation  of  Scores  in  Part  I 
to  Scores  in  Part  II  in  Terms  of  Percentiles  for  Design  Engineers  (+) 
and  for  Sales  Engineers  (o). 

exceptions  as  seen  from  the  positions  of  the  circles  and  crosses 
on  the  scatter  diagram ;  but  investigation  shows  reasonable  causes 
for  these,  which  do  not  disprove  the  hypothesis.  The  test  was 
made  for  graduate  student  engineers  in  the  Educational  Depart- 
ment of  the  Westinghouse  Company,  and  all  these  engineers  are 
always  just  graduated  from  the  technical  schools.  Therefore, 
they  would  not  have  the  difficulty  of  having  specialized  in  in- 
dustry for  several  years  and  having  forgotten  some  of  their  theor- 
etical or  technical  training.  Some  of  the  design  engineers  who 
took  this  test  had  been  doing  very  highly  specialized  work  for 
five  years,  and  had  forgotten  some  of  the  information  in  other 
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lines  of  electrical  engineering.  For  this  reason,  they  may  not 
have  done  so  well  in  Part  II,  although  they  were  regarded  by  the 
Chief  Engineer  as  among  his  best  engineers,  particularly  because 
of  their  experience,  conscientious  work,  and  constant  applica- 
tion. However,  I  do  not  think  that  the  specialization  or  ex- 
perience of  the  design  engineers  nor  of  the  sales  engineers  ma- 
terially affected  their  scores ;  for  Part  II  is  a  test  of  engineering 
aptitude  and  not  of  engineering  information,  and  Part  I  is  a 
general  intelligence  or  mental  alertness  test.  The  correlation 
between  Part  I  and  Part  II  was  r  =  +.42  for  the  design  en- 
gineers, and  r  =  +.84  for  the  sales  engineers.  If  the  design 
engineers  have  special  capacity  for  engineering,  as  we  believe 
they  have,  it  is  to  be  expected  that  their  score  in  Part  II,  which 
tests  engineering  capacity,  would  correlate  less  with  their  scores 
in  Part  I,  which  is  a  general  intelligence  test,  than  the  sales  en- 
gineers' scores  in  Part  II  would  correlate  with  their  scores  in 
Part  I.  The  scatter  diagram  of  the  scores  of  the  design  en- 
gineers in  Part  II  shows  that  the  correlation  between  the  parts  is 
lower  because  some  of  the  design  engineers  had  special  aptitude  in 
engineering,  but  could  not  do  well  in  the  general  intelligence  or 
mental  alertness  test. 

(f)  Statistical  Interpretation  of  the  Results  for  Occupational 
Placement.' — The  test  is  to  be  used  in  two  different  ways  for  two 
different  purposes.  The  first  use  is  to  separate  the  men  who  are 
best  fitted  to  be  sales  engineers  from  those  who  are  best  fitted 
to  be  design  engineers.  What  is  desired  in  this  case  is  to  know 
for  what  the  men  are  best  fitted  by  their  special  abilities,  not  con- 
sidering their  general  intelligence  or  mental  alertness.  A  man 
might  be  more  successful  as  a  design  engineer  and  might  make 
a  better  score  in  Part  II  than  another  man ;  but  even  so  he  would 
not  be  so  well  placed  if  he  is  placed  as  a  design  engineer  when 
he  could  be  still  more  successful  as  a  sales  engineer.  Each  man 
should  be  placed  not  where  he  would  be  merely  successful,  but 
where  he  could  function  most  efficiently  and  be  most  successful. 
Therefore  a  single  score  in  a  sales  test  or  in  an  engineering  test 
could  not  be  relied  upon  to  place  the  men ;  for  the  good  all-around 
man  would  do  best  in  both  kinds  of  work,  and  best  in  both  kinds 
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of  tests.  However,  some  measure  of  the  kinds  of  abilities  or 
the  qualitative  differences  in  intelligence  could  be  obtained  by- 
subtracting  the  score  of  each  man  in  one  test  from  his  score  in 
the  other  test.  Suppose,  for  each  individual,  we  subtracted  the 
score  of  Part  II  in  terms  of  the  standard  deviation  from  the 
score  of  Part  I  in  terms  of  the  standard  deviation.  The  man 
who  had  a  score  in  Part  I  which  was  considerably  higher  than 
his  score  in  Part  II  would  have  a  large  algebraically  positive  dif- 
ference between  his  scores,  and  he  would  be  a  sales  engineer  type 
of  man.  This  great  difference  between  his  scores  in  each  part  of 
the  test  might  be  because  he  did  well  in  Part  I,  or  did  poorly  in 
Part  II,  or  for  both  reasons.  In  any  case,  he  would  be  better 
fitted  for  sales  engineering  than  he  would  be  adapted  for  design 
engineering.  The  man  who  had  a  score  in  Part  I  considerably 
lower  than  his  score  in  Part  II,  would  have  a  large  algebraically 
negative  difference  between  his  scores ;  and  he  would  be  a  design 
engineer  type  of  man.  This  great  difference  between  his  scores 
in  each  part  of  the  test  might  be  because  he  did  poorly  in  Part  I, 
or  did  well  in  Part  II,  or  for  both  causes.  By  use  of  this  method, 
that  is,  transmuting  the  scores  into  terms  of  sigmas  and  sub- 
tracting the  score  in  Part  II  from  the  score  in  Part  I,  those  who 
get  the  greater  algebraically  positive  differences  are  to  be  con- 
sidered the  more  specially  fitted  for  sales  engineering;  and  those 
who  get  the  greater  negative  differences  are  to  be  considered  the 
more  specially  fitted  for  design  engineering.  The  practical  use 
of  this  method  is  to  rank  the  men  according  to  the  algebraical 
differences  between  their  respective  scores  in  Part  I  and  Part  II, 
and  then  count  down  the  list  until  the  desired  number  of  sales 
engineers  is  obtained.  That  is,  the  man  who  did  well  in  Part  I, 
but  poorly  in  part  II,  would  rank  high  in  the  list  according  to  the 
algebraic  difference  between  his  score  in  Part  I  and  his  score  in 
Part  II,  and  he  would  be  chosen  for  sales  engineering.  This 
method,  theoretically,  selects  the  men  who  can  function  most 
efficiently  as  sales  engineers,  and  leaves  those  who  can  function 
most  efficiently  as  design  engineers. 

To  check  this  method  of  classifying  and  ranking  the  thirty 
design  engineers  and  the  fifty-nine  sales  engineers,  the  score  of 
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each  individual  for  Part  II  was  subtracted  from  his  score  in  Part 
I,  both  scores  being  in  terms  of  their  respective  standard  devia- 
tions. This  gave  positive  although  not  the  most  desirable  results 
in  separating  the  sales  engineers  from  the  design  engineers. 
These  results  are  shown  graphically  in  Diagram  20.     Consider- 
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Diagram  20.  Differentiation  Obtained  by  Subtracting  Scores  in  Part  I  from 
Scores  in  Part  I  in  Terms  of  Standard  Deviations. 

Considering  the  diagram  as  a  four-fold  table,  the  engineers  indicated  by  the  test 
as  sales  engineers  are  79  per  cent  correctly  so  classified ;  and  the  engineers  indi- 
cated by  the  test  as  design  engineers  are  49  per  cent  correctly  so  classified. 
ru  =  +  45- 

ing  the  diagram  as  a  four-fold  table,  this  method  of  interpreting 
the  results  differentiates  the  men  so  that  of  those  classed  as  sales 
engineers,  79  per  cent  are  correctly  so  classed ;  and  out  of  those 
classed  as  design  engineers,  49  per  cent  are  correctly  so  classed. 
The  correlation  between  the  test  results  treated  in  this  way  and 
the  actual  classification  was  ru  =  +  .45.  Also,  the  correlation 
between  the  classification  of  the  men  by  the  test  treated  in  this 
way  and  the  classification  of  the  men  by  the  "Section  II.  Choice 
of  Other  Occupations"  of  the  Record  of  Interests,  was  r  =  +.50. 
A  similar  and  simpler  method  of  treating  the  results  is  to  trans- 
mute the  raw  scores  into  percentile  ranks,  and  subtract  the  per- 
centile rank  in  Part  II  from  the  percentile  rank  in  Part  I.  This 
was  done,  and  the  classification  of  the  engineers  was  practically 
the  same;  for  the  correlation  between  differences  in  percentile 
ranks  (Part  I  minus  Part  II  and  the  differences  in  scores  in  terms 
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of   sigma    (Part   I   minus   Part   II    in   terms   of   sigmas)    was 

r  =  +-97- 

The  best  method  of  dealing  with  the  results  is  to  divide  the 

score  in  Part  I  by  the  score  in  Part  II  for  each  individual.  This 
is  the  method  which  is  the  simplest,  and  it  is  the  one  which  gives 
the  most  efficient  use  of  the  test  as  a  means  of  differentiating 
the  men.  This  method  magnifies  the  differences  among  the  in- 
dividuals, which  differences  are  shown  by  showing  that  the  ability 
of  each  individual  to  deal  with  Part  I  differs  from  his  ability 
to  deal  with  Part  II.  That  is,  the  ratio  or  quotient  of  a  man's 
score  in  Part  I  divided  by  his  score  in  Part  II  changes  geo- 
metrically, instead  of  algebraically,  as  this  man's  special  ability 
is  greater  or  less.  This  probably  makes  the  results  agree  more 
nearly  with  the  practical  considerations ;  for  the  extreme  cases  of 
special  ability  should  be  markedly  differentiated.  The  genius 
is  valued  probably  far  greater  than  an  actual  measurement  of 
capacity  would  indicate.  The  results  of  dealing  with  the  scores 
by  this  method  are  presented  graphically  in  Diagram  2 1 .    Consid- 
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in  Part  II. 

Considering  the  diagram  as  a  four-fold  table,  the  engineers  indicated  by  the  test 
as  sales  engineers  are  81  per  cent  correctly  so  classified ;  and  the  engineers  indi- 
cated by  the  test  as  design  engineers  are  63  per  cent  correctly  so  classified. 
ru  =  +-7o. 
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ering  the  diagram  as  a  four-fold  table,  we  have  the  engineers 
differentiated  as  follows : 

Knowing  first  the  actual  classification  of  the  engineers,  we  can 
say  that  63  per  cent  of  the  design  engineers  were  shown  by  the 
test  to  be  better  fitted  for  design  engineering  than  81  per  cent 
of  the  sales  engineers  were  shown  to  be.  81  per  cent  of  the 
sales  engineers  were  shown  by  the  test  to  be  better  fitted  for  sales 
engineering  than  63  per  cent  of  the  design  engineers  were  shown 
to  be.  Not  knowing  first  the  classification  of  a  group  of  en- 
gineers, this  test  can  be  used  to  differentiate  them  as  follows : 
Assuming  that  if  an  engineer's  score  in  Part  I  of  the  test  is  more 
than  3.2  times  his  score  in  Part  II,  he  is  a  sales  engineer,  and 
that  if  his  score  in  Part  I  is  less  than  3.2  times  his  score  in  Part 
II,  he  is  a  design  engineer  or  a  pure  type  of  engineer,  this  test 
divides  the  engineers  so  that  81  per  cent  of  those  above  this  crit- 
ical ratio  (Part  I  at  least  as  much  as  3.2  times  Part  II)  are  sales 
engineers ;  and  63  per  cent  of  those  below  it  are  design  engineers 
or  of  a  purely  engineering  type.  (This  critical  ratio  may  be 
changed  slightly  by  the  change  from  the  mimeographed  form  of 
the  test  to  the  printed  form.)  The  correlation  between  the  test 
results  treated  in  this  way  and  the  actual  classification  of  the 
men  were  ru  =  +  -7°-  Also,  the  correlation  between  the  classifi- 
cation of  the  men  by  the  test  results  treated  in  this  way  and  the 
classification  of  the  men  by  the  Record  of  Interests,  Section  II, 
was  r  =  +46.  The  correlation  between  the  marks  for  the  en- 
gineers when  the  scores  were  treated  in  this  way  (score  in  Part 
I  divided  by  score  in  Part  II)  and  the  marks  for  the  engineers 
when  the  score  in  part  II  was  subtracted  from  the  score  in  Part 
I  in  terms  of  sigmas,  was  r  =  +.50.  Such  was  the  correlation 
coefficient  obtained  when  the  quotients  (Part  I  divided  by  Part 
II)  were  correlated  with  differences  between  scores  (Part  I 
minus  Part  II  in  terms  of  sigmas)  by  the  Pearson  product-mo- 
ment formula.  However,  this  does  not  show  the  true  relation- 
ship; for  the  relationship  is  a  non-linear  one;  and  by  using  the 
formula  for  eta,  it  is  shown  to  be  y  =  .88. 
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The  second  use  of  the  test  is  to  rank  the  sales  engineers  and 
the  design  engineers  after  they  have  been  separated,  so  as  to  show 
who  are  the  best  design  engineers,  and  who  are  the  best  sales 
engineers,  considering  general  intelligence  or  all-around  mental 
ability  as  well  as  special  ability.  In  other  words,  this  use  of  the 
test  makes  it  a  general  intelligence  or  mental  alertness  test.  The 
relationships  of  the  results  of  the  test  with  the  estimates  of  the 
success  of  the  engineers  have  already  been  given  in  the  pre- 
vious section.  There  the  composite  of  Part  I  and  Part  II 
was  obtained  by  simply  adding  the  scores  of  the  two  parts  in 
terms  of  their  standard  deviations.  The  composite  does  not 
necessarily  show  a  higher  correlation  with  the  estimates  of  suc- 
cess than  each  part  separately  shows;  for  if  the  two  parts  test 
two  widely  different  abilities,  then  combination  may  not  give  a 
higher  correlation  with  the  criterion.  The  simplest  method  of 
combining  the  scores  is  to  add  the  percentile  rank  of  each  man  in 
Part  I  to  his  percentile  rank  in  Part  II,  and  then  rank  the  men 
according  to  these  sums  of  percentile  ranks.  This  gives  results 
which  are  practically  as  reliable  as  those  obtained  by  the  method 
of  combining  scores  in  terms  of  sigmas. 

On  the  basis  of  the  results  obtained  so  far,  the  test,  both  Part 
I  and  Part  II,  is  being  given  to  the  graduate  engineers  who  are 
entering  the  employ  of  the  particular  industrial  firm.  Only  by 
following  up  these  men  in  the  lines  of  work  which  they  enter 
later,  will  we  finally  know  the  value  of  the  rating  scales,  of  the 
Record  of  Interests,  and  of  this  special  aptitude  test. 

The  results  already  obtained  on  one  hundred  seven  engineers 
employed  by  the  firm  during  the  first  year  that  the  test  has  been 
used  are  very  similar  to  the  results  obtained  on  the  group  studied 
in  this  research.  These  hundred  and  seven  men  have  been  with 
the  company  approximately  one  year  and  they  have  been  classi- 
fied according  to  previous  methods  into  sales  engineers  and  de- 
sign engineers  by  the  executives  supervising  their  work.     The 
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test  results  were  not  a  factor  in  the  classification.  However, 
assuming  that  the  executives  classified  the  men  correctly  at  the 
end  of  a  year,  we  can  say  that  the  test  results  alone  would  have 
classified  seventy  per  cent  of  the  men  correctly  on  the  day  that 
they  were  employed.  Beginning  with  June,  1921,  the  company 
will  use  the  test  and  other  methods  recommended  here  as  definite 
aids  in  classifying  and  placing  the  graduate  engineers  whom  they 
employ. 

PART  III.    SUMMARY  OF  RESULTS  AND  CONCLU- 
SIONS AND  RECOMMENDATIONS 

X.     Summary  of  Results 

The  practical  problem  upon  which  we  began  work  was  to 
formulate  methods  and  means  based  upon  sound  psychological 
principles  for  differentiating  graduate  engineers.  The  results 
of  studying  various  existing  methods  and  certain  proposed  or 
newly  devised  ones  showed  that  some  were  of  value  and  that 
others  were  not.  The  results  might  be  summarized  briefly  as 
follows : 

1.  Technical  school  grades  when  coming  from  various 
schools  of  different  standards,  do  not  differentiate  the  men  in  a 
large  group  according  to  their  ability  in  industry,  nor  according 
to  the  kind  of  work  which  they  can  do  best.  This  is  with  the 
one  exception  that  men  who  are  selected  to  be  design  engineers  are 
the  kind  of  men  who  made  higher  grades  in  technical  schools. 
This  was  true  in  all  subjects  except  in  general  academic  subjects, 
particularly  economics,  in  which  the  design  engineers  fell  below 
their  own  average,  and  the  grades  of  the  sales  engineers  were 
above  their  own  average,  so  that  the  grades  of  the  two  groups 
were  practically  the  same  in  these  subjects. 

2.  Neither  the  grades  given  in  the  classes  of  the  Educational 
Department  nor  the  grades  given  in  the  shop  on  the  courses  of 
the  Educational  Department  show  any  consistency  or  reliability. 
The  correlation  between  the  two  sets  of  grades  is  only  r  =  +.19, 
and  the  correlation  is  zero  with  all  other  measures  such  as  tech- 
nical school  grades,  ratings,  and  test  scores.    The  unreliability  of 
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these  grades  and  ratings  is  doubtless  due  to  the  method  of  making 
the  ratings,  of  recording  them,  and  of  combining  them,  rather 
than  being  altogether  due  to  the  inability  of  the  foremen  to  make 
the  ratings. 

3.  The  ratings  made  by  the  foremen  on  the  ten  different  traits 
or  qualifications  show  no  agreement  with  any  other  measures 
such  as  shop  and  class  grades,  ratings,  technical  school  grades,  or 
test  scores.  There  is  high  intercorrelation  among  the  traits. 
This  is  due  to  two  causes :  (a)  Some  of  the  traits  are  very  sim- 
ilar, or  at  least  they  are  not  well  enough  defined  for  the  foremen 
to  distinguish  them ;  and  as  the  foremen  are  not  trained  to  make 
such  ratings,  they  decide  that  a  man  is  either  a  good  man  or  a 
bad  man,  and  then  rate  the  man  accordingly  either  good  or  bad 
in  all  traits,  (b)  The  method  of  making  the  ratings,  in  terms  of 
A,  B,  or  C,  does  not  give  the  foremen  any  scale  or  basis  of 
measure  or  comparison. 

4.  The  only  traits  rated  by  the  foremen  which  differentiate 
the  different  groups  of  graduate  student  engineers,  are  tact, 
initiative,  enthusiasm,  and  personality.  The  trait  called  per- 
sonality is  the  most  vague  and  undefined,  but  it  shows  the  greatest 
differentiation  of  the  men ;  for  it  is  a  trait  concerning  which  the 
foremen  really  get  an  impression  and  form  an  opinion  in  regard 
to  each  man. 

5.  The  Interviewer's  Rating  Scale  as  already  in  use  by  the 
Company,  has  given  results  that  have  significant  correlation  with 
test  scores,  with  later  estimate  of  intelligence,  and  with  technical 
school  grades.  The  trait  called  Personal  Qualities,  as  previously 
defined,  had  a  high  correlation  with  other  traits,  and  was  too 
broad,  inclusive,  and  ambiguous  to  give  reliable  results.  It  was 
then  more  narrowly  defined  so  as  not  to  be  ambiguous  and  not 
overlap  or  include  the  other  traits  to  be  rated. 

6.  The  College  Instructor's  Rating  Scale  was  prepared  to  en- 
able the  college  instructors,  professors,  or  deans  to  record  a 
definite  estimate  of  a  senior  on  specific  traits.  This  is  expected 
to  give  more  reliable  and  significant  ratings  of  a  man  than  a 
verbal  expression  of  opinion  of  the  man  in  general. 

7.  The  Shop  and  Class  Rating  Scale  was  prepared  to  take 
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the  place  of  the  method  of  rating  on  ten  traits  by  A,  B,  and  C. 
It  requires  ratings  on  a  few  very  significant  traits  or  qualities ;  but 
it  is  expected  to  enable  the  foremen  and  the  instructor  to  have  a 
common  understanding  of  just  what  is  meant  by  these  traits  and 
to  give  a  reliable  and  significant  rating  that  will  differentiate  the 
engineers  according  to  essential  qualifications. 

8.  The  use  of  the  Record  of  Interests  shows  that  men  in  dif- 
ferent lines  of  work  have  different  interests.  That  is,  the  insur- 
ance Salesmen  as  a  group,  have  always  had  interests  different 
from  either  sales  engineers  or  design  engineers.  The  sales  en- 
gineers are  men  who  have  been  more  interested  in  economic, 
public,  and  social  affairs.  The  design  engineers  are  men  who 
have  been  more  interested  in  science,  machinery,  and  actual  con- 
struction or  fabrication  of  things. 

9.  The  engineers  and  the  salesmen  showed  a  definite  tendency 
to  like  or  be  interested  in  occupations  which  in  nature  of  work 
were  similar  to  those  they  were  already  following.  The  kind  of 
occupations  which  they  thought  would  give  them  the  greatest 
satisfaction  to  follow,  was  a  significant  criterion  of  the  kind  of 
work  in  which  they  could  be  and  were  already  successful. 

By  computing  for  each  engineer  the  percentage  which  his 
choices  of  occupations  of  a  sales  nature  were  of  his  total  number 
of  choices,  a  definite  measure  of  his  sales  engineering  interest  as 
opposed  to  design  engineering  interest  was  obtained.  By  this 
measure,  78  per  cent  of  the  sales  engineers  were  of  a  sales  type ; 
and  82  per  cent  of  the  design  engineers  were  of  an  engineering 
type.  Or  assuming  that  we  did  not  know  the  actual  occupations 
of  the  engineers,  those  engineers  which  this  test  of  interests 
alone  would  select  for  sales  engineering,  would  be  89  per  cent 
correctly  placed  or  classified;  and  the  men  which  the  test  of  in- 
terests selected  for  design  engineering  would  be  68  per  cent  cor- 
rectly placed  or  classified. 

10.  Personnel  Bureau  Test  VI,  a  general  intelligence  test 
similar  to  the  Army  Alpha  intelligence  test,  gave  results  that  did 
not  differentiate  the  engineers  according  to  the  kinds  of  work  for 
which  they  were  thought  to  be  fitted.  The  only  men  who  had 
scores  significantly  different  from  the  other  engineers,  were  the 
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design  engineers.  They  were  a  highly  selected  group  of  men  who 
had  shown  ability  in  the  engineering  classes  of  the  Company's 
Educational  Department. 

ii.  The  general  intelligence  test  gave  scores  that  correlated, 
r  =  +.48,  with  intelligence  as  estimated  by  two  executives  of 
the  Educational  Department  who  were  intimately  associated  with 
the  graduate  student  engineers  one  year. 

12.  The  general  intelligence  scores  correlated,  r  =  +.38, 
with  technical  school  grades,  and  with  interview  ratings  at  the 
time  of  employment,  r=  +-58;  but  the  correlations  were  zero 
with  shop  and  class  ratings,  which  did  not  correlate  with  any 
other  criteria,  and  were  evidently  unreliable. 

13.  Test  10,  A  Test  for  Graduate  Student  Engineers,  gave 
positive  although  not  perfect  results  in  separating  the  sales  en- 
gineers from  the  design  engineers.  Knowing  first  the  actual 
classification  of  the  engineers,  we  can  say  that  63  per  cent  of  the 
design  engineers  were  shown  by  the  test  to  be  better  fitted  for 
design  engineering  than  81  per  cent  of  the  sales  engineers  were 
shown  to  be.  81  per  cent  of  the  sales  engineers  were  shown  by 
the  test  to  be  better  fitted  for  sales  engineering  than  63  per  cent  of 
the  design  engineers  were  shown  to  be.  Not  knowing  first  the 
classification  of  a  group  of  engineers,  this  test  can  be  used  to  dif- 
ferentiate then  as  follows :  Assuming  that  if  an  engineer's  score 
in  Part  I  of  the  test  is  more  than  3.2  times  his  score  in  Part  II, 
he  is  a  sales  engineer,  and  that  if  his  score  in  Part  I  is  less  than 
3.2  times  his  score  in  Part  II,  he  is  a  design  engineer  or  of  a 
pure  engineering  type,  this  test  divides  the  engineers  so  that  81 
per  cent  of  those  above  this  critical  ratio  (Part  I  3.2  times  Part 
II)  are  sales  engineers,  and  63  per  cent  of  those  below  it  are  de- 
sign engineers  or  of  a  purely  engineering  type.  (This  critical 
score  may  be  changed  slightly  by  a  change  from  the  mimeo- 
graphed form  of  the  test  to  the  printed  form.) 

14.  The  differentiation  of  the  engineers  resulting  from  the 
use  of  Test  10  showed  a  comparatively  high  correlation  with 
the  results  of  the  Record  of  Interests.  The  correlation  was 
r  =  +.50  when  the  scores  in  Part  II  were  subtracted  from  those 
in  Part  I  in  terms  of  sigmas.     The  correlation  was  r  =  +.46 
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when  the  differentiation  was  obtained  by  dividing  the  scores  in 
Part  I  by  the  scores  in  Part  II.  This  shows  not  only  that  there 
is  high  correlation  between  interest  and  special  ability,  but  also 
that  the  Record  of  Interests  and  Test  10  give  reliable  results. 

15.  Test  10  used  as  a  general  intelligence  or  mental  alertness 
test  for  sales  engineers  gave  scores  that  correlated,  r  =  +.86 
and  +.81  with  estimated  success.  The  same  test  used  as  a  gen- 
eral intelligence  test  for  design  engineers  gave  scores  that  cor- 
related, r  — +.22  and  +.59,  with  estimated  success. 

XI.     Conclusions  and  General  Principles  Derived 

from  the  Study 

1.  The  study  of  the  technical  school  grades  show  that  such 
grades  cannot  be  made  of  practical  use  to  differentiate  engineers 
for  various  kinds  of  work,  except  in  a  very  general  way.  Sales 
engineers  do  comparatively  better  in  economics  than  in  engineer- 
ing subjects.  The  design  engineers  make  much  lower  grades  in 
economics  than  they  do  in  mathematics  and  science,  engineering, 
and  shop. 

2.  When  foremen  are  asked  to  rate  men  under  them  on  a 
large  number  of  traits  which  are  not  carefully  defined,  the  ratings 
are  unreliable  and  of  little  use  as  a  measure  of  the  value  of  the 
men  rated ;  and  such  ratings  are  particularly  of  no  use  for  classi- 
fying the  men  and  placing  them  in  positions  for  which  they  are 
best  fitted.  Generalizing,  we  can  say  that  ratings  made  by  dif- 
ferent people  on  traits  which  are  understood  by  some  to  be  one 
thing  and  by  others  to  be  another,  are  of  no  value.  This  difficulty 
arises  if  the  traits  are  not  carefully  defined  in  terms  that  the 
raters  can  understand;  and  there  is  the  same  difficulty  if  there 
are  many  traits  to  be  rated,  some  of  which  overlap  or  include 
others  so  that  they  can  not  be  differentiated.  The  ratings  are  un- 
reliable if  there  is  no  scale  or  standard  with  which  the  men  can 
be  compared.  This  basis  of  comparison  may  be  the  other  men  of 
the  same  group ;  but  there  must  be  definite  comparison  instead  of 
arbitrary  grades  or  marks.  Finally,  the  ratings  or  marks  must 
be  recorded  in  such  a  form  that  they  can  be  readily  tabulated  and 
actually  used  in  placing  the  men. 
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4.  Carefully  prepared  acquaintance-comparison  rating  scales, 
such  as  the  Interviewer's  Rating  Scale  and  the  Shop  and  Class 
Rating  Scale,  give  much  more  reliable  results  than  forms  on 
which  arbitrary  grades  or  marks  are  recorded  opposite  certain 
names  of  traits.  The  acquaintance-comparison  rating  scale  en- 
ables the  rater  to  compare  the  one  to  be  rated  with  other  people 
as  concrete  examples  of  different  grades  of  the  same  trait. 

5.  The  use  of  the  Record  of  Interests  shows  that  men  in  dif- 
ferent lines  of  work  have  different  interests.  The  interests  of  a 
person  are  not  in  just  one  specific  occupation,  but  they  are  general 
to  the  extent  that  they  pertain  to  very  similar  or  closely  allied  oc- 
cupations or  activities.  Interest  in  a  certain  class  of  activities  is 
a  criterion  that  the  person  will  be  interested  in  any  other  very  sim- 
ilar activity.  By  similar  activity  is  meant  one  that  requires  much 
the  same  information,  training,  experience,  kind  of  materials  and 
tools  worked  with,  mental  activity,  personality,  ideals  of  accuracy 
and  perfection,  and  social  attitude. 

6.  In  the  engineering  profession,  men  cannot  be  differentiated 
for  different  lines  of  work  by  tests  which  measure  what  is  com- 
monly known  as  general  intelligence  or  mental  alertness.  Men 
in  the  same  kind  of  engineering  work  differ  in  this  intelligence 
as  greatly  as  men  in  different  kinds  of  engineering  work.  The 
differences  which  fit  these  men  for  different  kinds  of  work,  are 
something  other  than  differences  in  intelligence. 

7.  General  intelligence  as  measured  by  a  general  intelligence 
test  does  show  a  significant  positive  correlation  with  the  success 
of  engineers  in  the  same  kind  of  work. 

8.  One  man  differs  from  another  in  having  special  abilities, 
which  functioning  in  an  integrated  form  as  a  kind  of  intelligence, 
can  be  measured  and  used  as  criteria  for  placing  the  man  in  the 
kind  of  work  for  which  he  is  best  fitted. 

9.  The  occupational  interests  of  a  man  show  a  definite  cor- 
relation with  the  kind  of  intelligence  or  special  abilities  which  he 
has,  and  with  the  kind  of  occupation  in  which  he  is  successful. 
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XII.     Specific  Methods  and  Practices  Recommended 

i.  Before  senior  engineers  who  are  being  graduated  from 
the  technical  schools,  are  employed,  they  should  be  rated  by  the 
use  of  rating  scales.  The  representatives  of  the  Company  who 
interview  the  seniors,  should  use  the  Interviezver's  Rating  Scale. 
Before  the  interviewers  start  on  their  visits  to  the  colleges,  they 
should  carefully  make  a  Master  Rating  Scale  from  the  list  of 
engineers  whom  they  know  intimately  and  who  have  passed 
through  the  Educational  Department  within  the  previous  five 
years.  If  possible,  two  or  three  interviewers  representing  the 
Company  should  interview  the  senior  and  make  ratings  independ- 
ently, and  then  combine  the  ratings  later. 

2.  After  the  interviewers  have  returned  to  the  home  office 
and  have  prepared  a  large  tentative  list  of  seniors  who  are  to  be 
considered  for  employment,  the  name  of  each  senior  should  be 
put  on  a  blank  College  Instructor's  Rating  Scale,  and  this  blank 
sent  to  the  dean,  professor,  or  instructor  who  can  be  depended 
upon  to  give  a  reliable  rating  of  the  man  under  consideration.  A 
rating  by  more  than  one  instructor  is  desirable  if  it  can  be  ob- 
tained without  imposing  on  the  instructors.  Most  of  the  pro- 
fessors are  glad  to  help  place  their  students.  If  the  representative 
believes  that  the  senior  will  be  desired  by  the  Company  and  that 
this  senior  will  make  application,  he  should  get  the  rating  from 
the  professor  while  visiting  the  college  or  as  soon  as  possible. 

3.  As  soon  as  the  graduate  student  engineers  enter  the  Edu- 
cational Department  of  the  Company,  they  should  be  under  ob- 
servation in  the  shop  and  in  the  class  to  be  rated  at  the  end  of  each 
month  by  means  of  the  Shop  and  Class  Rating  Scale. 

4.  Within  the  first  two  months  after  the  graduate  student 
engineer  enters  the  Educational  Department  of  the  Company,  he 
should  be  given  the  Test  10,  A  Test  for  Graduate  Student  En- 
gineers. This  should  be  scored,  first  to  determine  for  what  kind 
of  work  the  engineer  is  probably  best  fitted;  second,  to  determine 
what  the  rank  in  general  intelligence  is. 

5.  At  the  end  of  two  months  or  just  before  it  is  necessary  to 
segregate  the  graduate  student  engineers  for  intensive  training  in 
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some  particular  line  of  work,  the  student  engineers  should  be 
requested  to  fill  out  the  blank  entitled  Record  of  Interests. 

6.  At  the  end  of  two  months  or  when  it  is  necessary  to  know 
into  what  line  of  work  the  student  engineers  are  going,  the  re- 
sults of  the  rating  scales,  the  test,  and  the  Record  of  Interests, 
should  be  combined.  The  impressions  and  opinions  concerning 
the  engineers  which  the  executives,  interviewers,  and  instructors 
of  the  Educational  Department  have  recorded  en  the  rating  scales, 
should  have  a  weight  or  vote  of  one-third  in  the  final  composite 
measure  of  the  men;  the  test  should  have  a  vote  or  weight  of  one- 
third  ;  and  the  Record  of  Interests  should  have  a  weight  or  vote 
of  one-third  to  determine  for  what  line  of  work  the  engineer 
should  be  recommended.  In  case  of  doubt,  any  two  of  these 
measures  or  criteria  should  determine  for  what  work  the  partic- 
ular man  should  be  recommended.  Of  course  the  judgments  of 
the  executives  concerned  will  be  the  final  criteria,  but  their  im- 
pressions and  opinions  should  be  based  upon  dependable  data  and 
be  corrected  by  this  data. 

7.  The  student  engineer  should  be  interviewed  with  his 
Record  of  Interests  before  him  and  the  interviewer;  and  in  the 
interview  the  student  should  be  advised  to  enter  the  line  of  work 
for  which  he  is  best  fitted  as  determined  by  this  Record  of  In- 
terests, the  test,  and  the  rating  scales.  This  advice  should  be 
given  as  an  intelligent  recommendation;  but  the  student  should 
not  be  urged  against  his  will  to  enter  a  particular  line  of  work. 
If  the  student  objects  to  the  recommendation,  it  should  be  dis- 
cussed in  the  light  of  the  results  of  the  Record  of  Interests  and  of 
the  test. 

8.  After  the  student  engineers  have  been  segregated  and  as- 
signed tentatively  to  their  respective  lines  of  work,  a  copy  of  the 
Record  of  Interests  and  the  results  of  the  test  should  be  trans- 
mitted to  the  heads  of  the  respective  departments  in  which  the 
men  will  work. 

9.  The  results  of  all  rating  scales,  tests,  and  Record  of  In- 
terests, should  be  carefully  recorded  and  filed;  and  the  later  suc- 
cess or  record  of  the  engineers  should  be  systematically  followed 


84  BRUCE  V.  MOORE 

up  in  order  to  check  the  reliability  and  value  of  the  methods  and 
means  of  selection,  classification,  and  placement  being  used. 

10.  Some  person  qualified  for  personnel  research  work  should 
be  assigned  the  responsibility  for  keeping  rating  scales  and  tests 
in  proper  use,  working  up  the  results,  carrying  on  the  research 
and  follow-up  work,  and  planning  and  directing  new  lines  of 
research  which  should  be  undertaken  to  increase  the  efficiency  of 
the  educational,  placement,  and  other  personnel  work. 

ii.  The  rating  scales,  tests,  and  Record  of  Interests  should  be 
studied  to  make  further  improvements  as  a  result  of  their  use.  A 
duplicate  or  alternate  form  of  the  test  should  be  prepared,  which 
could  be  given  in  case  there  is  reason  to  believe  there  has  been 
coaching  for  the  first  form. 

12.  In  the  light  of  results  obtained  in  the  study  of  this  prob- 
lem with  graduate  student  engineers,  research  work  in  personnel 
problems  should  be  extended  to  the  apprentices,  the  clerical  work- 
ers and  the  shop  employees  of  the  Company. 
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